July 2010

Volume 25 Number 07

Cutting Edge - Expando Objects in C# 4.0

By Dino Esposito | July 2010

Most of the code written for the Microsoft .NET Framework is based on static typing, even though .NET supports dynamic typing via reflection. Moreover, JScript had a dynamic type system on top of the .NET 10 years ago, as did Visual Basic. Static typing means that every expression is of a known type. Types and assignments are validated at compile time and most of the possible typing errors are caught in advance.

The well-known exception is when you attempt a cast at run time, which may sometimes result in a dynamic error if the source type is not compatible with the target type.

Static typing is great for performance and for clarity, but it’s based on the assumption that you know nearly everything about your code (and data) beforehand. Today, there’s a strong need for relaxing this constraint a bit. Going beyond static typing typically means looking at three distinct options: dynamic typing, dynamic objects, and indirect or reflection-based programming.

In .NET programming, reflection has been available since the .NET Framework 1.0 and has been widely employed to fuel special frameworks, like Inversion of Control (IoC) containers. These frameworks work by resolving type dependencies at run time, thus enabling your code to work against an interface without having to know the concrete type behind the object and its actual behavior. Using .NET reflection, you can implement forms of indirect programming where your code talks to an intermediate object that in turn dispatches calls to a fixed interface. You pass the name of the member to invoke as a string, thus granting yourself the flexibility of reading it from some external source. The interface of the target object is fixed and immutable—there’s always a well-known interface behind any calls you place through reflection. 

Dynamic typing means that your compiled code ignores the static structure of types that can be detected at compile time. In fact, dynamic typing delays any type checks until run time. The interface you code against is still fixed and immutable, but the value you use may return different interfaces at different times.

The .NET Framework 4 introduces some new features that enable you to go beyond static types. I covered the new dynamic keyword in the May 2010 issue. In this article, I’ll explore the support for dynamically defined types such as expando objects and dynamic objects. With dynamic objects, you can define the interface of the type programmatically instead of reading it from a definition statically stored in some assemblies. Dynamic objects wed the formal cleanliness of static typed objects with the flexibility of dynamic types.

Scenarios for Dynamic Objects

Dynamic objects are not here to replace the good qualities of static types. Static types are, and will remain for the foreseeable future, at the foundation of software development. With static typing, you can find type errors reliably at compile time and produce code that, because of this, is free of runtime checks and runs faster. In addition, the need to pass the compile step leads developers and architects to take care in the design of the software and in the definition of public interfaces for interacting layers.

There are, however, situations in which you have relatively well-structured blocks of data to be consumed programmatically. Ideally, you’d love to have this data exposed through objects. But, instead, whether it reaches you over a network connection or you read it from a disk file, you receive it as a plain stream of data. You have two options to work against this data: using an indirect approach or using an ad hoc type.

In the first case, you employ a generic API that acts as a proxy and arranges queries and updates for you. In the second case, you have a specific type that perfectly models the data you’re working with. The question is, who’s going to create such an ad hoc type?

In some segments of the .NET Framework, you already have good examples of internal modules creating ad hoc types for specific blocks of data. One obvious example is ASP.NET Web Forms. When you place a request for an ASPX resource, the Web server retrieves the content of the ASPX server file. That content is then loaded into a string to be processed into an HTML response. So you have a relatively well-structured piece of text with which to work.

To do something with this data, you need to understand what references you have to server controls, instantiate them properly and link them together into a page. This can be definitely done using an XML-based parser for each request. In doing so, though, you end up paying the extra costs of the parser for every request, which is probably an unacceptable cost.

Due to these added costs of parsing data, the ASP.NET team decided to introduce a one-time step to parse the markup into a class that can be dynamically compiled. The result is that a simple chunk of markup like this is consumed via an ad hoc class derived from the code-behind class of the Web Forms page:

<html>
<head runat="server">
  <title></title>
</head>
<body>
  <form id="Form1" runat="server">
    <asp:TextBox runat="server" ID="TextBox1" /> 
    <asp:Button ID="Button1" runat="server" Text="Click" />
    <hr />
    <asp:Label runat="server" ID="Label1"></asp:Label>
  </form>
</body>
</html>

Figure 1 shows the runtime structure of the class created out of the markup. The method names in gray refer to internal procedures used to parse elements with the runat=server elements into instances of server controls.

Figure 1 The Structure of a Dynamically Created Web Forms Class

You can apply this approach to nearly any situation in which your application receives external data to process repeatedly. For example, consider a stream of XML data that flows into the application. There are several APIs available to deal with XML data, from XML DOM to LINQ-to-XML. In any case, you have to either proceed indirectly by querying the XML DOM or LINQ-to-XML API, or use the same APIs to parse the raw data into ad hoc objects.

In the .NET Framework 4, dynamic objects offer an alternative, simpler API to create types dynamically based on some raw data. As a quick example, consider the following XML string:

<Persons>
  <Person>  
    <FirstName> Dino </FirstName>
    <LastName> Esposito </LastName>
  </Person>
  <Person>
    <FirstName> John </FirstName>
    <LastName> Smith </LastName>
  </Person>  
</Persons>

To transform that into a programmable type, in the .NET Framework 3.5 you’d probably use something like the code in Figure 2.

Figure 2 Using LINQ-to-XML to Load Data into a Person Object

var persons = GetPersonsFromXml(file);
foreach(var p in persons)
  Console.WriteLine(p.GetFullName());

// Load XML data and copy into a list object
var doc = XDocument.Load(@"..\..\sample.xml");
public static IList<Person> GetPersonsFromXml(String file) {
  var persons = new List<Person>();

  var doc = XDocument.Load(file);
  var nodes = from node in doc.Root.Descendants("Person")
              select node;

  foreach (var n in nodes) {
    var person = new Person();
    foreach (var child in n.Descendants()) {
      if (child.Name == "FirstName")
        person.FirstName = child.Value.Trim();
      else
        if (child.Name == "LastName")
          person.LastName = child.Value.Trim();
    }
    persons.Add(person);
  }

  return persons;
}

The code uses LINQ-to-XML to load raw content into an instance of the Person class:

public class Person {
  public String FirstName { get; set; }
  public String LastName { get; set; }
  public String GetFullName() {
    return String.Format("{0}, {1}", LastName, FirstName);
  }
}

The .NET Framework 4 offers a different API to achieve the same thing. Centered on the new ExpandoObject class, this API is more direct to write and doesn’t require that you plan, write, debug, test and maintain a Person class. Let’s find out more about ExpandoObject.

Using the ExpandoObject Class

Expando objects were not invented for the .NET Framework; in fact, they appeared several years before .NET. I first heard the term used to describe JScript objects in the mid-1990s. An expando is a sort of inflatable object whose structure is entirely defined at run time. In the .NET Framework 4, you use it as if it were a classic managed object except that its structure is not read out of any assembly, but is built entirely dynamically.

An expando object is ideal to model dynamically changing information such as the content of a configuration file. Let’s see how to use the ExpandoObject class to store the content of the aforementioned XML document. The full source code is shown in Figure 3.

Figure 3 Using LINQ-to-XML to Load Data into an Expando Object

public static IList<dynamic> GetExpandoFromXml(String file) { 
  var persons = new List<dynamic>();

  var doc = XDocument.Load(file);
  var nodes = from node in doc.Root.Descendants("Person")
              select node;
  foreach (var n in nodes) {
    dynamic person = new ExpandoObject();
    foreach (var child in n.Descendants()) {
      var p = person as IDictionary<String, object>);
      p[child.Name] = child.Value.Trim();
    }

    persons.Add(person);
  }

  return persons;
}

The function returns a list of dynamically defined objects. Using LINQ-to-XML, you parse out the nodes in the markup and create an ExpandoObject instance for each of them. The name of each node below <Person> becomes a new property on the expando object. The value of the property is the inner text of the node. Based on the XML content, you end up with an expando object with a FirstName property set to Dino.

In Figure 3, however, you see an indexer syntax used to populate the expando object. That requires a bit more explanation.

Inside the ExpandoObject Class

The ExpandoObject class belongs to the System.Dynamic namespace and is defined in the System.Core assembly. ExpandoObject represents an object whose members can be dynamically added and removed at run time. The class is sealed and implements a number of interfaces:

public sealed class ExpandoObject : 
  IDynamicMetaObjectProvider, 
  IDictionary<string, object>, 
  ICollection<KeyValuePair<string, object>>, 
  IEnumerable<KeyValuePair<string, object>>, 
  IEnumerable, 
  INotifyPropertyChanged;

As you can see, the class exposes its content using various enumerable interfaces, including IDictionary<String, Object> and IEnumerable. In addition, it also implements IDynamicMetaObjectProvider. This is the standard interface that enables an object to be shared within the Dynamic Language Runtime (DLR) by programs written in accordance with the DLR interoperability model. In other words, only objects that implement the IDynamicMetaObjectProvider interface can be shared across .NET dynamic languages. An expando object can be passed to, say, an IronRuby component. You can’t do that easily with a regular .NET managed object. Or, rather, you can, but you just don’t get the dynamic behavior.

The ExpandoObject class also implements the INotifyPropertyChanged interface. This enables the class to raise a PropertyChanged event when a member is added or modified. Support of the INotifyPropertyChanged interface is key to using expando objects in Silverlight and Windows Presentation Foundation application front ends.

You create an ExpandoObject instance as you do with any other .NET object, except that the variable to store the instance is of type dynamic:

dynamic expando = new ExpandoObject();

At this point, to add a property to the expando you simply assign it a new value, as below:

expando.FirstName = "Dino";

It doesn’t matter that no information exists about the FirstName member, its type or its visibility. This is dynamic code; for this reason, it makes a huge difference if you use the var keyword to assign an ExpandoObject instance to a variable:

var expando = new ExpandoObject();

This code will compile and work just fine. However, with this definition you’re not allowed to assign any value to a FirstName property. The ExpandoObject class as defined in System.Core has no such member. More precisely, the ExpandoObject class has no public members.

This is a key point. When the static type of an expando is dynamic, the operations are bound as dynamic operations, including looking up members. When the static type is ExpandoObject, then operations are bound as regular compile-time member lookups. So the compiler knows that dynamic is a special type, but does not know that ExpandoObject is a special type.

In Figure 4, you see the Visual Studio 2010 IntelliSense options when an expando object is declared as a dynamic type and when it’s treated as a plain .NET object. In the latter case, IntelliSense shows you the default System.Object members plus the list of extension methods for collection classes.

Figure 4 Visual Studio 2010 IntelliSense and Expando Objects

It should also be noted that some commercial tools in some circumstances go beyond this basic behavior. Figure 5shows ReSharper 5.0, which captures the list of members currently defined on the object. This doesn’t happen if members are added programmatically via an indexer.

Figure 5 The ReSharper 5.0 IntelliSense at Work with Expando Objects

To add a method to an expando object, you just define it as a property, except you use an Action<T> or Func<T> delegate to express the behavior. Here’s an example:

person.GetFullName = (Func<String>)(() => { 
  return String.Format("{0}, {1}", 
    person.LastName, person.FirstName); 
});

The method GetFullName returns a String obtained by combining the last name and first name properties assumed to be available on the expando object. If you attempt to access a missing member on expando objects, you’ll receive a RuntimeBinderException exception. 

XML-Driven Programs

To tie together the concepts I’ve shown you so far, let me guide you through an example where the structure of the data and the structure of the UI are defined in an XML file. The content of the file is parsed to a collection of expando objects and processed by the application. The application, however, works only with dynamically presented information and is not bound to any static type.

The code in Figure 3 defines a list of dynamically defined person expando objects. As you’d expect, if you add a new node to the XML schema, a new property will be created in the expando object. If you need to read the name of the member from an external source, you should employ the indexer API to add it to the expando. The ExpandoObject class implements the IDictionary<String, Object> interface explicitly. This means you need to segregate the ExpandoObject interface from the dictionary type in order to use the indexer API or the Add method:

(person as IDictionary<String, Object>)[child.Name] = child.Value;

Because of this behavior, you just need to edit the XML file to make a different data set available. But how can you consume this dynamically changing data? Your UI will need to be flexible enough to receive a variable set of data.

Let’s consider a simple example where all you do is display data through the console. Suppose the XML file contains a section that describes the expected UI—whatever that means in your context. For the purpose of example, here’s what I have:

<Settings>
    <Output Format="{0}, {1}" 
      Params="LastName,FirstName" /> 
  </Settings>

This information will be loaded into another expando object using the following code:

dynamic settings = new ExpandoObject();
  settings.Format = 
    node.Attribute("Format").Value;
  settings.Parameters = 
    node.Attribute("Params").Value;

The main procedure will have the following structure:

public static void Run(String file) {
    dynamic settings = GetExpandoSettings(file);
    dynamic persons = GetExpandoFromXml(file);
    foreach (var p in persons) {
      var memberNames = 
        (settings.Parameters as String).
        Split(',');
      var realValues = 
        GetValuesFromExpandoObject(p, 
        memberNames);
      Console.WriteLine(settings.Format, 
        realValues);
    }
  }

The expando object contains the format of the output, plus the names of the members whose values are to be displayed. Given the person dynamic object, you need to load the values for the specified members, using code like this:

public static Object[] GetValuesFromExpandoObject(
  IDictionary<String, Object> person, 
  String[] memberNames) {

  var realValues = new List<Object>();
  foreach (var m in memberNames)
    realValues.Add(person[m]);
  return realValues.ToArray();
}

Because an expando object implements IDictionary<String, Object>, you can use the indexer API to get and set values.

Finally, the list of values retrieved from the expando object are passed to the console for actual display. Figure 6 shows two screens for the sample console application, whose only difference is the structure of the underlying XML file.

Figure 6 Two Sample Console Applications Driven by an XML File

Admittedly, this is a trivial example, but the mechanics required to make it work are similar to that of more interesting examples. Try it out and share your feedback!

Dino Esposito is the author of “Programming ASP.NET MVC” from Microsoft Press and coauthor of “Microsoft .NET: Architecting Applications for the Enterprise” (Microsoft Press, 2008). Based in Italy, Esposito is a frequent speaker at industry events worldwide.

Thanks to the following technical expert for reviewing this article: Eric Lippert