Debugging

DataTips, Visualizers and Viewers Make Debugging .NET Code a Breeze

Morgan Skinner

Download the Code Sample

There is a host of new goodies in Visual Studio® 2005 to make your coding more productive. These include generics, anonymous methods in C#, Master Pages in ASP.NET, and various new controls in Windows® Forms, to name just a few. Underpinning these advancements is another batch of improvements which are less likely to grab the limelight but are nonetheless very important. Regardless of how good a programmer you are or how many flashy tools are available in your toolbox, you will always have to jump into the debugger at some point in the process to see what's going wrong. The only problem with breaking into the debugger in such a scenario is that the debugging environment isn't currently as rich in features as it could be.

As an example, suppose you have an application that retrieves some data from a database and stores it locally within a DataSet. Could you go through a DataSet within the debugger and tell exactly what's in it? How about after some modifications? Can you tell which rows have been updated, and what the old and new values are? This scenario is common because there is often some object within the debugger which is not simple to decompose manually. Luckily, Visual Studio 2005 comes to the rescue with a raft of new debugger features. There are DataTips, which are nicely formatted tooltip windows, displayed while debugging, that can expand to drill into more detail on a particular object. The next level of enhancement comes with Type Viewers, which consist of a number of new attributes that can alter the display of a particular type within the current debug windows. At the top of the pile are Type Visualizers, which allow you to write a custom viewer window for a particular type.

In this article, I'll briefly discuss DataTips, and then go on to discuss Type Viewers and Type Visualizers in more depth.

DataTips and Type Viewers

When you're debugging, you often want to explore a particular variable to see what it contains. As an example, consider an Employee class, which might contain several member variables and properties. A sample C# definition of such a class is shown in Figure 1.

Figure 1 Employee Class

using System;
public class Employee
{
    public Employee(int employeeID, string forename, string surname)
    {
        _employeeID = employeeID;
        _forename = forename;
        _surname = surname;
    }
    public int EmployeeID
    {
        get
        {
            return _employeeID;
        }
    }
    public string FullName
    {
        get
        {
            return string.Format("{0} {1}", _forename, _surname);
        }
    }
    public string Forename
    {
        get
        {
            return _forename;
        }
        set
        {
            _forename = value;
        }
    }
    public string Surname
    {
        get
        {
            return _surname;
        }
        set
        {
            _surname = value;
        }
    }
    private int _employeeID;
    private string _forename;
    private string _surname;
}

A DataTip is displayed within the code window when the mouse hovers over a particular variable. The data displayed is a dynamic, expandable tooltip, which expands as the mouse is moved over the display. Figure 2 shows the display of data for an object as the mouse hovers over the variable.

A DataTip
Figure 2 A DataTip

This display is similar to that shown within the Quick View or Watch windows, but it's now possible to get this display without switching away from the code window. So far all of this is for free—DataTips just work, so any current classes you have will exhibit this behavior in the debugger.

The displayed data, however, contains both the internal class members and public properties, which produces a confusing display as data is replicated. If you could somehow filter the data to show only necessary items then that would be a real step in the right direction. Also, displaying "{Employee}" is rather crude. It would be better to be able to control what's displayed for a given item to something under the programmer's control. For both of these facilities, you need to look at Type Viewers.

With the DataTip shown in Figure 2, there is too much information, so you need some way to show only the public properties. This customized view will be used in DataTips and also Watch windows and the Quick Watch window. To get this to work, you must attribute your type using one or more of the new Debugger attributes. In the following discussion, I'll describe these attributes and show the effect of these within the Watch window.

DebuggerDisplayAttribute

This attribute is used to provide a single-line summary of a type within the data windows of the debugger. The text displayed is now under your complete control, so that the following could be used on the employee type to display the FullName property:

[DebuggerDisplay("Employee ({FullName})")] 
public class Employee { 
    ••• 
}

The debugger uses this string as a format specification, and replaces items within curly braces with the corresponding data from the object being displayed. Within the braced expression, you get implicit access to the object's this pointer. The data within the braces can be a field, property, or method, and can have any visibility, so using private members here is permitted. The syntax examples shown in Figure 3 would result in the output shown in Figure 4. Note that I've constructed this image from three different runs to show them all in one image. The use of expressions within the DebuggerDisplay attribute should be kept to a minimum, as the expansion of the attribute is done by the debugger and is language specific.

Figure 3 Debugger Output

Syntax Description
[DebuggerDisplay(  "{_forename} {_surname}")] Uses the private _forename and _surname members
[DebuggerDisplay(  "Employee- {ToString()}")] The ToString method is called to provide the textual representation to be displayed as the value of the object
[DebuggerDisplay("Employee  ( {Forename} {(_wizard) ?  \"Is a Wizard\" : \"Is   \"Is not a Wizard\" } )")] An expression is evaluated in order to provide a differing representation to the user based on the value of a flag

 

Types in the Debugger 
Figure 4 Types in the Debugger

The DebuggerDisplay attribute allows you to customize the display of a given object. However, when the object is expanded you're still rewarded with private data and properties. There are two ways to change what is displayed when a given type is expanded. The first is to attribute some fields as being invisible to the debugger, and the second is to use a proxy class.

DebuggerBrowsableAttribute

DebuggerBrowsableAttribute is added to members of a given class to specify whether they are visible, and if so, what state they should show up in. The attribute has one parameter, that being a value from the DebuggerBrowsableState enumeration. The chart in Figure 5 describes the four possible values for this enumeration, and describes the output that is displayed in the data window when that value is used on a member.

Figure 5 DebuggerBrowsableState Values

Value Description
Collapsed The item will be shown initially collapsed within the data window. This is the same as the default behavior of the debugger.
Expanded This will expand a hierarchical type when its containing type is expanded. If you had a Customer class with an internal list of Orders, then that list of orders could be automatically expanded for the user when the Customer was expanded.
Never This attribute hides the member so that it is not displayed at all within the debugger data windows. It would be used in my Employee class to hide the internal members, such as _forename and _surname.
RootHidden This value indicates that the attributed member should not be shown, but should instead be expanded to show submembers.

 

With these attributes, it is possible to create a simple display for a given type, but there may be cases where simply hiding fields isn't enough. This is where the DebuggerTypeProxy attribute is useful.

DebuggerTypeProxy

When added to a type, the DebuggerTypeProxy attribute specifies a separate class used as a proxy when displaying that type. You can think of this attribute as a custom view for a type. It is common to define the proxy for a type as an internal nested class, since this kind of class has full access to all private details of the containing type and, as such, can use these details when constructing the display.

The proxy class must have a public constructor which accepts one argument of the type that is displayed by the proxy. All public properties of the proxy will be used in constructing the display and any private or protected variables will not be shown.

The code in Figure 6 shows a typical Address class, which contains fields for five address lines and a ZIP code. The proxy class has been designed to output only those address lines that are non-null. Note that the DebuggerTypeProxy is defined on the outer Address class.

Figure 6 Address Class

[DebuggerTypeProxy(typeof(AddressProxy))]
public class Address
{
    public Address(string line1, string line2,
        string line3, string line4,
        string line5, string zip)
    {
        _addressLine1 = line1;
        _addressLine2 = line2;
        _addressLine3 = line3;
        _addressLine4 = line4;
        _addressLine5 = line5;
        _zipcode = zip;
    }
    private string _addressLine1;
    private string _addressLine2;
    private string _addressLine3;
    private string _addressLine4;
    private string _addressLine5;
    private string _zipcode;
    internal class AddressProxy
    {
        public AddressProxy(Address address)
        {
            _address = address;
        }
        public string Address
        {
            get
            {
                StringBuilder sb = 
                   new StringBuilder(_address._addressLine1);
                if (null != _address._addressLine2)
                    sb.AppendFormat(", {0}", _address._addressLine2);
                if (null != _address._addressLine3)
                    sb.AppendFormat(", {0}", _address._addressLine3);
                if (null != _address._addressLine4)
                    sb.AppendFormat(", {0}", _address._addressLine4);
                if (null != _address._addressLine5)
                    sb.AppendFormat(", {0}", _address._addressLine5);
                if (null != _address._zipcode)
                    sb.AppendFormat(", {0}", _address._zipcode);
                return sb.ToString();
            }
        }
        private Address _address;
    }
}

The resulting display within the Watch window is shown in Figure 7. In addition to showing the simplified view, it is also possible to expand the Raw View node to see all of the internal members of the object. I'll discuss the magnifying glasses later.

Filtered Addresses 
Figure 7 Filtered Addresses

There are many classes in the Framework marked with the DebuggerTypeProxy attribute; it is especially useful for complex classes in which there are a large number of internal member variables that aren't usually vital to have when debugging. The view displayed is somewhat customizable in that with the addition of a proxy you can decide which data to display and which to hide. However when displaying more rich types such as DataSets and Images, it would be better to be able to "see" the object.

When debugging in Visual Studio, it's common to stop on a breakpoint and look through a number of local variables to see what they contain. A problem arises when that data does not have a simple representation, such as a bitmap or other complex data structure. In this situation, the debugger still presents a textual view of the data type, which at best can be used to find information such as the pixel dimensions of a bitmap, but won't really help with viewing the actual contents of that bitmap. What's needed is some way to graphically display an item in the debugger so that a custom window can display the data. It is now possible to perform this custom rendering in Visual Studio 2005 using Type Visualizers.

Type Visualizers

A Type Visualizer is a class in the Microsoft® .NET Framework that implements the IDebugVisualizer interface, which I'll discuss shortly. An assembly-scoped attribute is used to identify that the assembly contains a custom visualizer and to provide information as to what type of visualizer it applies to and the name displayed in the UI. I'll show an example of this attribute later in this article.

Visualizable Types
Figure 8 Visualizable Types

Any type that has a visualizer defined is displayed with a magnifying glass and dropdown menu, as shown in Figure 8. Clicking the magnifying glass will display the last chosen visualizer for that type. A small number of visualizer classes ship with the .NET Framework, most notably for the string class. There are three styles available for strings: a multiline text box showing the text, a version which displays the string as XML, and another that displays the string as HTML within a browser. These displays provide a much richer environment and should save a great deal of time when debugging, especially when you're constructing large segments of HTML or XML manually. The display shows a menu which lists all visualizers defined for the selected type (in this case a Hashtable, listing the visualizer from this article). The user can then click this menu item to display a modal UI.

Writing a Visualizer

A visualizer usually consists of three classes: one that implements the IDebugVisualizer interface, another that provides the user interface, and a third that serializes the .NET type from the debugee process to the debugger, since you need to copy the object from the debugee to the debugger in order to construct the visualization (you don't necessarily need to copy the whole object—just the necessary data). The code in Figure 9 is a minimal implementation of a visualizer for a Hashtable.

Figure 9 Hashtable Visualizer

using System;
using System.Diagnostics;
using System.Drawing;
using System.Windows.Forms;
[assembly: DebuggerVisualizer(
    typeof(HTVisualizer.MinimalVisualizer),
    VisualizerUIType.Modal,
    Target = typeof(System.Collections.Hashtable),
    Description = "Minimal Hashtable")]
namespace HTVisualizer
{
    /// <summary> /// Constructs a minimal implementation of a visualizer </summary> 
    public class MinimalVisualizer : IDebugVisualizer
    {
        void IDebugVisualizer.Show(
            IServiceProvider provider,
            IVisualizerObjectProvider objectProvider,
            VisualizerUIType uiType)
        {
            Form blank = new Form();
            blank.Text = "Put some controls on here!";
            IModalDialogVisualizerService modalService = provider.GetService(
                typeof(IModalDialogVisualizerService)) as IModalDialogVisualizerService;
            modalService.ShowDialog(blank);
        }
    }
}

Writing visualizers in this way means that they can be added to classes that already exist within the .NET Framework and are hooked up to the class they apply to at run time. The interfaces used for a visualizer are defined within the System.Diagnostics namespace, but they are buried within another assembly currently called meehost which ships with Visual Studio and is installed in the Common7\IDE directory. By the time Visual Studio 2005 ships, this assembly will be renamed and installed in the Global Assembly Cache (GAC).

In Figure 9 the DebuggerVisualizer attribute is applied with assembly scope and indicates that there is a visualizer which targets the Hashtable and is of type MinimalVisualizer. The Description property is used as the caption on the dropdown menu within Visual Studio. The code then requests the IModalDialogVisualizerService from the passed service provider, which is used in this example to show a blank form to the user.

The VisualizerUIType enumeration defines three values: Modal, DropDown, and ToolWindow. The latter two are not currently supported by the Visual Studio IDE but may well be implemented in a later version of the product.

Once the visualizer is written, it needs to be installed so that Visual Studio can access it within a debug session. In the present version, the assembly that contains the DebuggerVisualizer attribute should be copied into the Common7\Packages\Debugger\attribcache directory. This directory is probed by the debugger each time a debug session is begun, so files dropped in here will be evaluated the next time the debugger runs.

Getting the Data

So far I've discussed how to construct a visualizer, but I haven't described how to get hold of the object that is being displayed in order to do something useful with it.

The IDebugVisualizer.Show method is passed an object that implements the IVisualizerObjectProvider, which is used to obtain this serialized object, as shown in the following code snippet: Hashtable hash = objectProvider.GetObject ( ) as Hashtable ; Under the covers, this operation is serializing an object from the process being debugged, transporting it into the memory space of the debugger, and then deserializing it for use within the visualizer. The opposite is also possible in that a visualizer can replace an object within the debugee by calling IVisualizerObjectProvider.SetObject. This facility opens up some wonderful possibilities for tools. For years, developers have been updating an object within the debugee manually in the data window, but now there's a way for you to write your own custom code to do it.

Suppose you're testing some boundary condition that breaks the application. Rather than wading through a bunch of screens in order to set things up correctly, you could write a visualizer that would load and save objects such as a serialized DataSet. Without altering the source code whatsoever, you could set a breakpoint, load in a DataSet from disk, and continue with your debug session.

Even though the class that implements IDebugVisualizer is called a visualizer, your code doesn't have to provide a UI for the user. You could dump the contents of the currently selected data type to disk, or you could validate a string against an XML schema. You might also want to take a snapshot of an object to look at it later, or possibly revert an object to some state stored on disk. Feel free to come up with your own creative ideas because debugging just became much more fun!

Being able to read from as well as write to the object in the debugee is a great capability, but now that you have the hashtable, what do you do with it?

Visualizing the Hashtable

The impetus for writing this article was a late night session I had with the debugger and a hashtable. I've written my own hashtables in the past and was eager to display the data graphically within a .NET Hashtable. Along the way I unearthed the inner workings of this data structure, which made writing the visualizer all the more interesting.

For those of you who don't spend time working on the internals of data structures, a brief recap on how hashtables work may be in order. The main idea of a hashtable is to provide access to a particular object based on a key value. Each object added to a hashtable is allocated a slot based on the hash of this key, where a hash is the output of a function that generates a number based on some input data and which always generates that same number when provided with the same input. To retrieve an object, all you need to know is the key value, which when rehashed will return the same slot within the data structure so the object can be retrieved quickly, usually in O(1) (pronounced "order one") time.

A typical hashtable is made up of a number of "buckets." When an item is inserted into a hashtable, a bucket is chosen based on the hash of the key used for that item. A bucket is often implemented as a linked list into which the new item is inserted. This is called collision chaining, as multiple items that hash to the same bucket are chained together with a list. When searching a linear hashtable, a sequential linear search is performed on the linked list found in the bucket selected by the hash. Figure 10 shows an example hashtable, which consists of four buckets and some entries.

Simple Hashtable 
Figure 10 Simple Hashtable

Here the buckets are represented on the left, and each is the head of a linked list of nodes. In this simplified example, when an entry is added to the hashtable, the bucket selected is calculated as the remainder when the Key value is divided by 4 (in C# this would be Key % 4). When a new entry is added to the hashtable, its bucket number is calculated and it is then added into the list. In the case of keys 4 and 12, both fall into the same bucket, which is called a collision and which leads to a chain of entries.

A hashtable that has a good spread of buckets can provide optimal lookup time for a node, but one that has many chained objects isn't going to provide good random access to nodes because once a bucket has been found, the linked list needs to be traversed to find the particular node, resulting in O(n) (pronounced "order-en") performance.

The .NET Hashtable isn't implemented in as simplistic a manner as the one just described, and it performs better than the example shown in Figure 10 because it uses a nonlinear rehashing algorithm. Simply put, when an item is inserted, a candidate bucket is chosen, which involves a calculation based on the hashcode. If the bucket is full, another calculation is performed, which effectively jumps to another bucket in the hashtable. This occurs until an empty bucket is found, at which point the inserted item is stored in this bucket. Figure 11 shows a hashtable with three entries on the left with no collisions. In the hashtable on the right, there is a collision with the third element of the array, so the hash_coll term is updated to indicate this collision.

Hashtable 
Figure 11 Hashtable

The value stored in the hash_coll entry is not a pointer but a discrete value that the algorithm uses to determine whether to continue searching for a free slot. The value will have the high bit set if there is a collision, whereas it will be clear if no collision has occurred. In Figure 11, when the final item is added into the hashtable, the third entry would have the collision bit set. Deleting entries is also simple with the .NET Hashtable because the algorithm simply updates the key to a predefined value in order to indicate that the slot is vacant.

In order to work out how Hashtable was implemented, I used Lutz Roeder's excellent Reflector tool, available for download at http://www.aisto.com/roeder/dotnet. I find this tool invaluable and use it practically every day.

So, armed with the knowledge of how the Hashtable class was implemented, the next step was to implement a proxy class that would read a hashtable and convert the internal representation into something that could be serialized and consumed by the debugger. For the purposes of this example visualizer, only the string representation of the key and value are being used, as this cuts down on the amount of data being passed from the debugee to the debugger, as well as being all that can be displayed by the visualizer.

In order to decompose the hashtable into data that could then be passed to the debugger, I needed to do a small amount of reflection. In most circumstances, I would argue against the use of reflection, however sometimes there simply is just no other way. In this instance, the hashtable contains an internal array which contains the bucket information described previously. The visualizer uses code to read the data from the buckets array and converts it into a representation that can then be serialized. A bucket is defined as seen in Figure 12, which shows a slightly modified display from the Reflector tool.

A Bucket in the Reflector 
Figure 12 A Bucket in the Reflector

Getting the values contained within a particular bucket is a two-step process. The first step is to retrieve the private buckets collection, which is returned as the Array type. Retrieving a particular bucket is then accomplished by calling the GetValue method on the Array, which returns an object from which further code can be used to retrieve individual fields from the bucket. This example code is shown in Figure 13. The code presented in the download uses the same method, but in a slightly different manner as it has been split into two different functions. One function retrieves the buckets array and the other function retrieves values from a given bucket.

Figure 13 Get Buckets Array

Array buckets; 
FieldInfo fi = _hashtable.GetType().GetField ( "buckets" , 
    BindingFlags.NonPublic | BindingFlags.Instance ) ; 
buckets = fi.GetValue ( _original ) as Array ; 
// Next get a particular bucket, based on the passed bucketNumber 
object bucket = buckets.GetValue ( bucketNumber ) ; 
// From the bucket, read the key FieldInfo 
info = bucket.GetType ( ).GetField ( "key" , 
    BindingFlags.Public | BindingFlags.Instance ) ; 
object key = info.GetValue ( bucket ) ; 
// Then read the value 
info = bucket.GetType ( ).GetField ( "val" , 
    BindingFlags.Public | BindingFlags.Instance ) ; 
object val = info.GetValue ( bucket ) ; 
// Lastly get the hash_coll 
info = bucket.GetType ( ).GetField ( "hash_coll" , 
    BindingFlags.Public | BindingFlags.Instance ) ; 
int hash_coll = (int)info.GetValue ( bucket ) ;

To read the value of a private instance variable, you need to obtain the FieldInfo object for the field using reflection and then call its GetValue method. Because reflection is being used instead of strongly typed access during development (which you can't use to access the private fields of another class), the code in the download contains considerable error checking.

Once all of the code to decompose the hashtable into constituent parts has been completed, the next decision is how best to represent that data so that it can be used by the visualizer. Armed with the knowledge of how the hashtable was implemented, I decided upon the scheme that is illustrated in Figure 14.

The Hashtable Deconstructed
Figure 14 The Hashtable Deconstructed

The left-hand column shows the hash value generated for each discrete key in the hashtable. There may be fewer hashes created than entries within the hashtable as two keys might well produce the same hashcode. Similarly, a given hashcode might lead to a bucket that contains an object with a different hash due to the probing algorithm used by the implementation.

The third line displays the buckets that are traversed when looking for the item with a key value of 4. The hashcode of 4 is computed, and is then used to index into the buckets collection. Here it finds an entry which does not match the key, so it is shown with a greyed out caption (to indicate a collision). The hashtable algorithm then re-indexes into the array based on the current position, which results in bucket 39. This has the appropriate key, which is indicated by the yellow caption.

You can see that looking up the node with a key value of 4 is slower than the other items, due to this collision with the node in bucket 23. The following code produces this (abridged) data: Hashtable ht = new Hashtable(); for(int loop = 0; loop < 20; loop++) { ht.Add (loop.ToString(), loop.ToString()); }

There are only 20 entries in this collection, and when I tested this I received four collisions. The number of collisions isn't easy to predict as it is dependent on the key values, the number of items in the hashtable, and the current size of the buckets array (which is dynamically resized based on its content). It is always best to size any collection before adding items into it, and the hashtable is no exception to this rule, even though it will dynamically change size.

The code that displays the hashtable is fairly lengthy and space doesn't permit me to display it all, so feel free to download it and dissect it as necessary. There are two vital sections: deconstructing the hashtable and displaying it within the debugger process.

Rendering the Visualizer

When deciding how to display the hashtable, I wanted to be able to display some of the useful parts of the hashtable without going into too much complexity. I chose the display in Figure 14 because it's easy to read and it displays problems with hash collisions immediately. During testing I wrote some code that would cripple any keys passed to the hashing function so that only 1 of 10 possible hashes was returned. This proved instructive of how many collisions will occur if a poor hashcode generator is used. This visualizer will therefore be helpful if you have created your own hashing functions, as you can see how well your hashing algorithm is partitioning the items.

A number of features of the control are worth looking into. First, the control supports zooming, so the user can use the mouse wheel to zoom in or out of the control at run time. This is very useful when there are a large number of items within the hashtable. Quite a bit of code is required to support zooming, so I wrapped this in a reusable class and derived the Hashtable visualizer from it.

Each object displayed on screen is an instance of a Node class I've created, containing details of the display element, such as the text displayed in the title and details section. When I first wrote the code, each node also contained brush and font objects. However when displaying a large hashtable with many objects, this became too memory-intensive, so I added in a behavior object.

This behavior object is shared among many nodes, and it contains information such as the gradient fill colors and fonts to use when rendering the nodes. When a node is rendered, it uses its behavior object to obtain the appropriate visual styles. Using this pattern has made the rendering process not only faster but also less memory hungry. The rendering of each node uses a graphics path that specifies the rounded corners, and each node is built up from a number of different operations. One way to make this process faster still would be to pre-render each different node style into an offscreen bitmap, blit these onto the screen, and then draw the text.

Conclusion

In this article I have described some of the new debugging features that are in Visual Studio 2005. The DataTips feature shows debugging information right in the code where it's needed, without requiring you to open another window or select the appropriate object within the watch window. Being able to expand an object here is a fantastic step forward, and the ability to filter the displayed information for the appropriate data by using the new debugger attributes is further evidence of how hard the debugger team has been working to make debugging much less of a chore.

In my opinion, the best new features in Visual Studio 2005 are the custom visualizers. Using them, you can now freely extend part of the debugger experience, and display custom views of objects that are live within your debug session.

The example code for this article provides a visualizer for the hashtable and is available from the link at the top of this article.


Morgan Skinner is an application development consultant for Microsoft in the UK, specializing in Visual C#, Windows Forms, and ASP.NET. He's been working with .NET since PDC 2000. His home page is http://www.morganskinner.com.