Discovering Dallas: Part 2
It’s been a hectic couple of weeks, so I haven’t had a chance until now to dig back into Microsoft’s project “Dallas.” If you read my previous blog post, you should have a good idea of what “Dallas” is all about and how relatively easy it will be to pull a vast variety of public data sources into your applications.
If you paid really, really close attention to that post though, you’ll notice that when I mentioned using the C# service classes (which you can generate for a given data service via the “Dallas” Developer Portal pictured to the right), I specifically called out Windows Forms, WPF, and ASP.NET clients, but not Silverlight. Why no Silverlight? Well, the reason is two-fold:
- All service access from Silverlight applications must be asynchronous. The C# proxy that is generated provides only a synchronous implementation.
- A Silverlight application must obey the web application sandbox model and so can only access the domain from which it was downloaded. There are techniques to overcome this, however.
It’s likely that by the time “Dallas” is officially released, the generated service classes will be Silverlight-friendly, but I figured it would be a good exercise to walk through what it takes to overcome the two constraints above. I hope too that it helps give a leg up to those of you wanting to play with Silverlight and “Dallas” today.
In this post, I’ll take a look at adding an asynchronous capability to the generated C# service classes – a prerequisite for Silverlight access and just a good thing to have in general. In a subsequent post, I’ll tackle accessing “Dallas” end-to-end with Silverlight.
C# Service Class Overview
Let’s start by looking at what gets generated when you select the Download C# service classes option from the “Dallas” Developer Portal. I’m going to use the crime statistics from Data.gov for this example, but the concepts are the same for the other providers.
For each unique service series (note, some of the providers, like the Associated Press, comprise multiple services or series), the C# service class will include a data transfer object (DTO) class and a service proxy class. For instance, with the Data.gov crime statistics, you get DataGovCrimeByCitiesItem and DataGovCrimeByCitiesService classes.
The service class exhibits a few properties, including the $accountKey and $uniqueUserID fields passed via the HTTP request header and paging parameters that can be optionally included in the URL query string. As for methods, there’s a constructor and a public Invoke method. It’s the Invoke method that initiates the synchronous access to the “Dallas” service and shapes the resulting XML feed into a generic List of the “item” class:
1: public List<DataGovCrimeByCitiesItem> Invoke(
2: System.String state, System.String city, System.String year)
3: {
4: // argument validation logic elided
5:
6: StringBuilder builder = new StringBuilder();
7: builder.Append(this.Uri);
8: builder = builder.Replace("{state}", state);
9:
10: // additional URL construction logic elided
11:
12: List<DataGovCrimeByCitiesItem> result =
13: new List<DataGovCrimeByCitiesItem>();
14: foreach (XElement element in
15: this.InvokeWebService(builder.ToString()))
16: {
17: XElement properties = element.Element(
18: XName.Get("content", "https://www.w3.org/2005/Atom"))
19: .Element(XName.Get("properties",
20: "https://schemas.microsoft.com/.../dataservices/metadata"));
21:
22: DataGovCrimeByCitiesItem item =
23: new DataGovCrimeByCitiesItem();
24:
25: XElement __temp_State = properties.Element(
26: XName.Get("State",
27: "https://schemas.microsoft.com/ado/.../dataservices"));
28: if (__temp_State != null &&
29: !string.IsNullOrEmpty(__temp_State.Value))
30: item.State = (System.String)
31: Convert.ChangeType(__temp_State.Value,
32: typeof(System.String));
33:
34: // additional XElement -> property logic elided
35:
36: result.Add(item);
37: }
38:
39: return result;
40: }
InvokeWebService (below), the private method invoked in Line 15 above, handles the actual web request, passing in the required $accountKey and $uniqueUserID headers and processing the response stream. It returns a list (Lines 23-24 below) of the entry elements in the feed back to the Invoke method, which iterates through that list to create the List<T> that is returned to the client. The client can then process the list of items as needed, for instance, assigning them to the DataSource property of a DataGridView.
1: private IEnumerable<XElement> InvokeWebService(string url)
2: {
3: if (url == null)
4: throw new ArgumentNullException("url");
5:
6: HttpWebRequest request =
7: (HttpWebRequest)HttpWebRequest.Create(url);
8: request.Headers.Add("$accountKey", this.AccountKey);
9: request.Headers.Add("$uniqueUserID",
10: this.UniqueUserID.ToString());
11:
12: XDocument xml = null;
13: using(HttpWebResponse response =
14: (HttpWebResponse)request.GetResponse())
15: {
16: using(StreamReader reader =
17: new StreamReader(response.GetResponseStream()))
18: {
19: xml = XDocument.Parse(reader.ReadToEnd());
20: }
21: }
22:
23: return xml.Root.Elements(
24: XName.Get("entry", "https://www.w3.org/2005/Atom"));
25: }
Adding Asynchronous Support
An asynchronous pattern implementation presumes the client will call one method to invoke the request, and when the request is complete a callback method will be invoked to handle the results and return the information to the caller. In the currently generated class, the Invoke method is what the client calls, but there is currently no callback mechanism.
To provide asynchronous invocation capability, we essentially need to split the implementation of the Invoke method (and indeed the InvokeWebService method) right down the middle. One method, an overloaded Invoke method, will handle the processing up to the point of making the service call. Another method, I’m calling it ProcessResponse (and will discuss it later on) will serve as a callback method when the service call completes; it will handle the logic to extract the data from the data service’s XML feed and return it back to the client application.
Let’s start by looking at the overloaded Invoke method below.
1: public void Invoke(
2: System.String state, System.String city, System.String year,
3: Action<List<DataGovCrimeByCitiesItem>> callback)
4: {
5: if (state == null)
6: throw new ArgumentNullException("state");
7:
8: StringBuilder builder = new StringBuilder();
9: builder.Append(this.Uri);
10:
11: builder = builder.Replace("{state}", state);
12:
13: builder.Append("?$format=atom10");
14:
15: if (this.SupportsPaging)
16: {
17: builder.Append("&$page=" + this.CurrentPage);
18: builder.Append("&$itemsPerPage=" + this.ItemsPerPage);
19: }
20:
21: if (city != null)
22: builder.Append("&city=" + city);
23: if (year != null)
24: builder.Append("&year=" + year);
25:
26: HttpWebRequest request =
27: (HttpWebRequest)HttpWebRequest.Create(
28: new Uri(builder.ToString()));
29: request.Headers.Add("$accountKey",
30: this.AccountKey);
31: request.Headers.Add("$uniqueUserID",
32: this.UniqueUserID.ToString());
33:
34: IAsyncResult ar = request.BeginGetResponse(ProcessResponse,
35: new AsyncState<DataGovCrimeByCitiesItem>()
36: {
37: Request = request,
38: Callback = callback
39: });
40: }
This version of Invoke starts out identical to the generated synchronous version, namely by building up the URL for the request. Next (Lines 26ff.) the request is created; in contrast, the synchronous version calls the private method InvokeWebService to do this part.
On lines 34-39, the asynchronous request is initiated via BeginGetResponse, which accepts two arguments:
ProcessResponse is the callback method that executes when the web request has completed; we’ll look at the code for that shortly.
AsyncState<T> is a class that encapsulates information about the asynchronous request. That class is one I’ve added, and it’s definition is as follows:
class AsyncState<T> { public WebRequest Request { get; set; } public Action<List<T>> Callback { get; set; } }
- The Request property retains the information about the HttpWebRequest object, so it can be referenced in the callback (ProcessResponse).
- The Callback property is a reference to a method that accepts a generic List (here it will be a list of DataGovCrimeByCitiesItem), as a parameter. The callback reference is passed as an argument into the Invoke method by the client and represents the client method that will be executed when the ProcessResponse method has completed. It’s essentially the mechanism whereby the results of the asynchronous call are returned to the invoker.
Now let’s take a look at ProcessResponse, the callback that executes when the asynchronous request in line 34 of Invoke above completes.
1: private void ProcessResponse(IAsyncResult ar)
2: {
3: AsyncState<DataGovCrimeByCitiesItem> asyncState =
4: (AsyncState<DataGovCrimeByCitiesItem>)ar.AsyncState;
5: HttpWebRequest request =
6: (HttpWebRequest)asyncState.Request;
7: HttpWebResponse response =
8: (HttpWebResponse)request.EndGetResponse(ar);
9:
10: XDocument xml = null;
11: IEnumerable<XElement> xmlElements = null;
12: using (StreamReader reader =
13: new StreamReader(response.GetResponseStream()))
14: {
15: xml = XDocument.Parse(reader.ReadToEnd());
16: xmlElements = xml.Root.Elements(
17: XName.Get("entry", "https://www.w3.org/2005/Atom"));
18: }
19:
20: List<DataGovCrimeByCitiesItem> result =
21: new List<DataGovCrimeByCitiesItem>();
22: foreach (XElement element in xmlElements)
23: {
24: XElement properties = element.Element(
25: XName.Get("content", "https://www.w3.org/2005/Atom"))
26: .Element(XName.Get("properties",
27: "https://schemas.microsoft.com/ado/.../dataservices/metadata"));
28:
29: DataGovCrimeByCitiesItem item =
30: new DataGovCrimeByCitiesItem();
31:
32: XElement __temp_State = properties.Element(
33: XName.Get("State",
34: "https://schemas.microsoft.com/ado/2007/08/dataservices"));
35: if (__temp_State != null &&
36: !string.IsNullOrEmpty(__temp_State.Value))
37: item.State = (System.String)Convert.
38: ChangeType(__temp_State.Value, typeof(System.String));
39:
40: // additional XElement -> property assignment elided
41:
42: result.Add(item);
43: }
44: asyncState.Callback(result);
45: }
- Lines 3 – 8 include standard code to get the state of the asynchronous request, whereby we regain access to the original web request and the callback method enveloped by the AsyncState<T> class.
- The majority of this code, specifically lines 10 through 43, is practically identical to that included in the original Invoke method. The only difference is that the source of the XML stream is now the response object versus the return value of the private InvokeWebService method.
- Line 44 is the mechanism through which the results, namely a list of DataGovCrimeByCitiesItem, are ‘sent’ back to the client application.
So the only thing missing at this point is the client invocation logic. Below is the implementation of a button click event on a WinForm window. The asynchronous request is made on line 6; here the method ProcessOutput is passed into the Invoke method and ultimately becomes the Callback property of the asyncState reference used in line 44 above.
ProcessOutput has access to the data returned from the request as a parameter, but since that request was executed on a non-UI thread, I can’t simply assign the result to the display element, here a DataGridView. To marshal the data back onto the UI thread, the Invoke method is used in line 11, along with a short lambda expression to carry out the data source assignment.
1: private void btnAsync_Click(object sender, EventArgs e)
2: {
3: DataGovCrimeByCitiesService svc = new DataGovCrimeByCitiesService(
4: "REDACTED",
5: Guid.NewGuid());
6: svc.Invoke("Rhode Island", null, "2007", ProcessOutput);
7: }
8:
9: public void ProcessOutput(List<DataGovCrimeByCitiesItem> data)
10: {
11: this.Invoke(
12: new Action(() => crimeGridView.DataSource = data), null);
13: }
A Working Example
Although it looks like a lot of code, most of it was just cut and paste, and there’s really only a couple of steps needed, and they are the same for any of the “Dallas” services you choose to access:
- Create an AsyncState<T> class (or just steal mine verbatim). Here the generic type parameter T represents the ‘item’ or data transfer object class that’s part of the original C# service class; in this example, it’s DataGovCrimeByCitiesItem.
- Create a new overloaded Invoke method in the service class file:
- Add a Action<List<T>> callback argument, and have the method return void.
- Copy the URL validation/manipulation code from the Invoke method of the existing service class.
- Add the code to make the asynchronous call (BeginGetResponse), using the AsyncState<T> class to retain the request and callback information.
- Create the callback method that executes when the asynchronous request completes (in my example, it’s ProcessResponse)
- Access the IAsyncResult information to get the response data; this is standard operating procedure for asynchronous invocation processing.
- Copy in the code to access the XML stream from the response object; in the original code, this is part of the synchronous InvokeWebRequest method.
- Copy the XML-to-Item processing loop from the original Invoke method.
- Invoke the client callback method at the end of the loop (replacing the
return result;of the original Invoke method).
- Create a method on the client that returns void and accepts a List<T>. This is the method that receives the data resulting from the asynchronous call and displays it in the client user interface. You’ll pass it as the last argument to the overloaded Invoke method created in step 2. Remember to marshal data appropriately to the UI thread!
Feel free to download the code I walked through here to your own machine and give it a whirl. The sample is a Windows Forms application that displays selected crime data in a DataGridView and supports both synchronous and asynchronous invocation. I’ve confined the service invocation changes to a new .cs file (AsyncProxyClasses.cs) that extends the partial class created by the generated proxies, so you’ll find all of the code there (well, except for the few lines needed to call it in the client WinForm).
Note though, that I’ve removed my $accountKey, so you won’t be able to run the application successfully until you’ve received your own key. You can request your own account key via the “Dallas” Developer Portal accessed from this link. Oh yeah, and as usual with demo code, proper exception handling is left to the reader :) You will, for instance get an HTTP 401 – Unauthorized exception if you try to run the code as is (without a valid $accountKey).