Cutting Edge
Context-Sensitive PictureBox Controls
Dino Esposito
Code download available at:CuttingEdge2006_07.exe(1940 KB)
Contents
The Trick Unveiled
Architecting a Richer PictureBox Control
Defining Hot Regions
Adding the Necessary Event Handling
Putting the Pieces Together
Data Binding Support
An ASP.NET ImageMap Control
Great ideas are timeless. A long time ago in Microsoft Systems Journal Paul DiLascia demonstrated a neat trick to display context-sensitive tooltips floating over pictures. As the user moved the mouse over the picture, the tooltip control updated its text to reflect the name of the pointed figure.
How can a tooltip control be smart enough to support a non-rectangular area that, for example, precisely follows the natural shape of a human figure in a painting? Hot spots in images are nothing new, but you normally define them through common, easy-to-describe shapes—rectangles, circles, or perhaps polygons.
Paul solved the issue brilliantly by loading two copies of the image—the original image and a hot spot map. The original photo is displayed in a picture-box component, and the map image is kept hidden and used to map each pixel of the original image to a particular color. The map image is nearly identical to the original except that it fills each hot spot area with a color. This way, each hot spot area corresponds to a unique color and each unique color can be linked to a tooltip text. Before the tooltip pops up, the pixel underneath the mouse is mapped to the corresponding pixel in the hot-spot map. The color on the hidden copy of the image is mapped to the hot-spot list and the related text finally pops up.
In this month’s installment of Cutting Edge, I’ll apply Paul’s trick to the standard Windows® Forms PictureBox control so that it can display context-sensitive tooltips and fire proper events when the user clicks on particular areas. This way, you can easily implement clickable maps using real-world pictures and enlarge or shrink them at your leisure with little effort.
The Trick Unveiled
The photo on the left in Figure 1 is displayed to the user; the one on the right is used under the hood to quickly determine whether any clicked pixel on the displayed image belongs to hot areas, which in this photo are red, lime green, cyan, magenta, and yellow. I have deliberately chosen a real-world picture to illustrate my point. You can use virtually any image you like and still be able to create a map of context-sensitive regions inside of it.
Figure 1** Image User Sees and Underlying Image with Hot Spots **
Let’s briefly examine a common scenario. Your user wants to click on a world map to select a given region and then examine sales figures. How would you detect clicking on the various regions? You can partition the map into polygons made by a sequence of points. Once you have a region, a graphic API in the Microsoft® .NET Framework can tell you whether a point belongs to it. This approach is common in ASP.NET applications where the ImageMap control makes implementing it relatively effortless. However, you’ll run into trouble if the final bitmap size changes somehow. As the map size changes, you need to recalculate all the points for all regions. Applying a scale factor may help, but a more flexible solution would be welcome.
There are two key benefits to using a second image. First, you can more carefully control the boundaries of each region. Second, editing regions is trivial—just a matter of coloring proper areas using a simple graphics editor, such as Microsoft Paint (see Figure 2).
Figure 2** Colored Hot Spots Defined on a World Map **
Using two images, especially in the context of Windows Forms applications, consumes negligible resources. In ASP.NET, memory consumption due to a double-image load is more critical. Fortunately, solutions can be architected to cache the image only once and share it among multiple requests and users.
The Windows Forms PictureBox control can display images in a variety of formats loaded from a variety of sources, including previously loaded Image objects, remote images at a known URLs, or images residing on disk. In the .NET Framework 2.0, PictureBox has been further enhanced to display a temporary picture while the control loads the main image and an error image when the selected image is not available.
In the .NET Framework 1.x, when the user clicks on the image, the control fires the Click event, but all that does is send notification of the user’s action; it provides no additional information about mouse position and so forth. Event handlers for the event receive a basic EventArgs:
Sub PictureBox1_Click(ByVal sender As Object, _ ByVal e As EventArgs) _ Handles PictureBox1.Click ... End Sub
In Windows Forms 2.0, you can take advantage of the new MouseClick event, which provides you with the position of the mouse at the time of the click:
Sub PictureBox1_MouseClick(ByVal sender As Object, _ ByVal e As MouseEventArgs) _ Handles PictureBox1.MouseClick ... End Sub
Similarly, if you want to display context-sensitive tooltips based on the portion of the image underneath the mouse, you can handle the MouseMove event from within the host form, process the position, and determine what to do.
Architecting a Richer PictureBox Control
The PictureBox control I’m going to implement in this column addresses both points. It accepts a collection of hot regions and fires client events if the user clicks on or moves over any such region. The overall programming interface mimics the programming interface of the ImageMap control in ASP.NET 2.0. The key difference is that the PictureBox control defines hot regions based on colors instead of points. The new PictureBox control derives from the Windows.Forms.PictureBox built-in control and features two essential properties: HotSpots and MapImage (see Figure 3).
Figure 3 Essential Properties of the PictureBox Control
Public Class PictureBox : Inherits Windows.Forms.PictureBox Private m_tooltip As ToolTip Private m_bmp As Bitmap Private m_hotSpots As New HotSpotElementCollection Public Sub New() InitializeAsSmartPictureBox() End Sub Public Property MapImage() As Image Get Return DirectCast(m_bmp, Image) End Get Set(ByVal value As Image) m_bmp = New Bitmap(value) End Set End Property <DesignerSerializationVisibility( DesignerSerializationVisibility.Content)> _ Public ReadOnly Property HotSpots() As HotSpotElementCollection Get Return m_hotSpots End Get End Property Private Sub InitializeAsSmartPictureBox() If Me.SizeMode = PictureBoxSizeMode.Normal Then m_tooltip = New ToolTip m_tooltipTitle = "You’re on" AddHandler Me.MouseClick, AddressOf OnClickInternal End If End Sub ... End Class
The MapImage property is of type Image and represents the companion image of the displayed picture where hot regions are drawn using a well-known palette of colors. The Image is wrapped by a Bitmap object—a standard GDI+ object—that exposes methods to get the color of a given pixel.
Look at the map in Figure 2. The Image property of the PictureBox will be bound to the topmost image; the new MapImage property will be associated with the bottom image. The two images must meet a few requirements. First, images must have the same size. Second, the companion image must be saved using a non-lossy format such as BMP or perhaps PNG. JPEG images should be avoided because they use color approximation in their compression. If the color of some pixels in a hot region is modified, the whole point detection mechanism breaks. GIF files are fine as long as the colors you use to mark hot regions are within the color palette. Otherwise, like with JPEG images there’s the risk of color approximation. The displayed image can be any format.
The Windows Forms PictureBox control features a SizeMode property that indicates how the image is displayed. Valid values for this property come from the PictureBoxSizeMode enumeration. The default is Normal, meaning that the image is rendered starting from the upper-left corner of the control, and any part of the image that exceeds the PictureBox’s area is clipped. The StretchImage value, on the other hand, causes the displayed image to stretch or shrink to fit the PictureBox’s size. This poses a critical issue given the expected behavior of the extended PictureBox control: the companion image should be resized by the same factor. This can be done programmatically using the GDI+ classes. For example, you can use the GetThumbnailImage method on the Image class or, better yet, the DrawImage method on the Graphics class. To calculate the ratio, you compare the size of the PictureBox to the size of the image. Note that there are good reasons to avoid using GetThumbnailImage in this scenario. It may have very low fidelity relative to the original image, and in rare scenarios it may actually be a completely different image. Though this is very unlikely, it would be quite difficult to diagnose. The sample PictureBox control won’t provide additional features if its size mode is anything but Normal. If this does not meet your needs, you can augment the sample code available for download from the MSDN®Magazine Web site.
To promptly catch any value changes in the SizeMode property, and enable additional capabilities, you need to handle the SizeModeChanged event in the control:
Protected Overrides Sub OnSizeModeChanged(ByVal e As EventArgs) MyBase.OnSizeModeChanged(e) m_beSmart = (Me.SizeMode = PictureBoxSizeMode.Normal) End Sub
An internal private member will track the working mode of the PictureBox. This member, the m_beSmart variable in the preceding code snippet, returns false if the SizeMode property is something other than Normal.
Defining Hot Regions
HotSpots is the second key property for the new PictureBox control. It is a collection of custom types, as you can see in Figure 4.
Figure 4 Defining Hot Spot Regions for the PictureBox Control
Imports System.Collections.ObjectModel Public Class HotSpotEventArgs : Inherits EventArgs Public HotSpot As HotSpotElement Public CancelTooltip As Boolean End Class Public Class HotSpotElement Private m_hotSpotColor As Color Private m_hotSpotID As Integer Private m_description As String Private m_title As String Public Property HotSpotColor() As Color Get Return m_hotSpotColor End Get Set(ByVal value As Color) m_hotSpotColor = value End Set End Property Public Property HotSpotID() As Integer Get Return m_hotSpotID End Get Set(ByVal value As Integer) m_hotSpotID = value End Set End Property Public Property Description() As String Get Return m_description End Get Set(ByVal value As String) m_description = value End Set End Property Public Property Title() As String Get Return m_title End Get Set(ByVal value As String) m_title = value End Set End Property Public Overrides Function ToString() As String Return String.Format("HotSpot: {0} ({1}-{2}-{3})", _ HotSpotID, HotSpotColor.R, HotSpotColor.G, HotSpotColor.B) End Function End Class Public Class HotSpotElementCollection Inherits Collection(Of HotSpotElement) Public Function ContainsColor(ByVal clr As Color) As Boolean For i As Integer = 0 To Me.Count - 1 Dim elem As HotSpotElement = Me.Item(i) If elem.HotSpotColor.ToArgb() = clr.ToArgb() Then Return True Next Return False End Function Public Function FindHotSpot(ByVal clr As Color) As HotSpotElement For i As Integer = 0 To Me.Count - 1 Dim elem As HotSpotElement = Me.Item(i) If elem.HotSpotColor.ToArgb() = clr.ToArgb() Then Return elem Next Return Nothing End Function Protected Overrides Sub InsertItem( _ ByVal index As Integer, ByVal item As HotSpotElement) If Not ContainsColor(item.HotSpotColor) Then MyBase.InsertItem(index, item) End If End Sub Protected Overrides Sub SetItem( _ ByVal index As Integer, ByVal item As HotSpotElement) If Not ContainsColor(item.HotSpotColor) Then MyBase.SetItem(index, item) End If End Sub End Class
HotSpots is of type HotSpotElementCollection, which is a generic type obtained from the Collection (Of T) type composed with the HotSpotElement type. The HotSpotElement class defines a region of color in a map image that is meaningful. In this context, a hot region is identified with a unique numeric ID, an RGB color, a title, and a description. The color is used to uniquely identify the region. If you’re interested in capturing user clicks, then the ID provides a numeric value to test on the client and determine what to do. It is likely that the color information would be enough to uniquely identify the clicked area; from this perspective, the ID information is redundant. However, it might be more useful than colors in rich data-binding scenarios where you want to associate regions on a map with the ID they have in the back-end database. In this way, the color information is not coupled with region information.
Title and description serve in another role as well—they can be used for context-sensitive tooltips that pop up as the user moves the mouse over the displayed picture.
You can populate the HotSpots property programmatically or declaratively from within the Visual Studio® 2005 designer. It’s nice that Visual Studio 2005 automatically recognizes collection properties and binds them to the built-in collection editor (see Figure 5).
Figure 5** Editing the Hotspots Collection Property **
The default collection editor adds collection members and displays them with any text that results from the member’s ToString method. In the rightmost grid, you see all the properties of the member type. Each property is editable as in the Visual Studio 2005 parent property grid.
If you’re familiar with ASP.NET control development, you know that changes entered through the designer are not persisted to the designer.vb or designer.cs codebehind file. If you don’t believe me, just try it. Drop the new PictureBox control on a form and populate the HotSpots collection. When you’re finished, save and start the form. The PictureBox control behaves as if the collection were empty, and the designer.vb file or the form has no hot spot elements saved. Why is that?
The answer to the problem is that you need to claim a particular designer serialization policy for the HotSpots collection property. In general, you need to do this for any collection properties in both Windows Forms and ASP.NET custom controls. The following attribute on the collection property does the trick:
<DesignerSerializationVisibility( _ DesignerSerializationVisibility.Content)>
Without this attribute no changes made to the collection at design time are ever persisted to the designer.vb file in Windows Forms or the ASPX source page in ASP.NET.
The HotSpotElementCollection type inherits from the generic Collection type and extends it with a couple of finder methods—ContainsColor and FindHotSpot. Both take a color as their input and return a Boolean value or a HotSpotElement object respectively, based on their findings. (Look back at Figure 4.) FindHotSpot, in particular, returns the HotSpotElement that matches the specified color, if any.
In addition, the HotSpotElementCollection class overrides a couple of methods—InsertItem and SetItem—to make sure that the color doesn’t conflict with something already in the collection.
Armed with full support for image hot regions, let’s proceed to implement a couple of events for the host form. I’ll define two events—HotSpotFound and HotSpotClicked. HotSpotFound fires when the mouse is moving over a hot spot; HotSpotClicked fires when the user clicks on a hot spot.
Adding the Necessary Event Handling
The HotSpotFound event is declared using the new generic version of the EventHandler type, as follows:
Public Event HotSpotFound As EventHandler(Of HotSpotEventArgs)
You saw the HotSpotEventArgs data structure back in Figure 4. It basically extends the base EventArgs class with a couple of properties—HotSpot and CancelTooltip. The HotSpot property references the hot spot area that has been found or clicked. The CancelTooltip property indicates whether, in case of mouse movements, the corresponding tooltip should be canceled. This property gives client code the final word on the tooltip displayed for the hot spot. By handling the event, the client form can programmatically change or even cancel the tooltip.
The HotSpotClicked event follows an identical schema.
Public Event HotSpotClicked As EventHandler(Of HotSpotEventArgs)
In this case, though, the CancelTooltip property is not used by the PictureBox control once the event handler returns. Whatever value you assign to the property in the event handler is blissfully ignored by the PictureBox control.
The PictureBox control registers its own handler for the MouseClick event upon instantiation, as you saw in Figure 3. The internal handler, OnClickInternal, is shown in Figure 6.
Figure 6 Firing the HotSpotClicked Event
Protected Overridable Sub OnClickInternal( _ ByVal sender As Object, ByVal e As MouseEventArgs) PrepareAndRaiseClickEvent(e.X, e.Y) End Sub Private Sub PrepareAndRaiseClickEvent( _ ByVal x As Integer, ByVal y As Integer) Dim elem As HotSpotElement = GetUnderlyingHotSpot(x, y) If elem Is Nothing Then HideTooltip() Return End If ‘ Raise an event to the host form with the underlying color Dim args As New HotSpotEventArgs args.HotSpot = elem RaiseEvent HotSpotClicked(Me, args) End Sub Private Function GetUnderlyingHotSpot( _ ByVal x As Integer, ByVal y As Integer) As HotSpotElement If Not m_beSmart Then Return Nothing ‘ Ensure the point is inside the bitmap If x >= m_bmp.Width Or y >= m_bmp.Height Then Return Nothing ‘ Get the underlying color and check it against HotSpot collection Return HotSpots.FindHotSpot(m_bmp.GetPixel(x, y)) End Function Private Sub HideTooltip() m_tooltip.SetToolTip(Me, String.Empty) End Sub
When a point in the client area of the PictureBox control is clicked, the control receives the client coordinates of the point from the event data structure—the MouseEventArgs class.
The next step entails finding the color of the pixel at the specified position. Note that the internal MouseClick event fires even when the user clicks outside the image’s boundaries. You need to catch these clicks and return without calling GetPixel on the Bitmap. If you call GetPixel with coordinates outside the image’s size, an exception is thrown. GetPixel is a method on the Bitmap class that returns the color of the pixel at the given location. You then take this color and look up for a region associated with that color. If any is found, the corresponding HotSpotElement object is returned:
Dim elem As HotSpotElement = HotSpots.FindHotSpot(clr)
You use the FindHotSpot method on the hot spot collection to retrieve information about the found region.
But, what if your image naturally contains pixels with the same color as one of your hot spot colors? If the user were to move the mouse exactly over that pixel, a conflict would occur. To avoid that, for hot regions you should ideally choose colors that are not used elsewhere in the image. With over 16 million colors available, finding an unused color is possible, but not easy. However, because of the way colored pixels are distributed in a real image, your color will probably be found in one pixel here and there and may not become a problem.
A more elegant solution would consist in reserving a color (white or transparent perhaps) for all parts of the image that do not have a region defined. In this way, you color uniformly all the underlying image except hot regions. The reserved color might be exposed as a public property to let developers choose it at will.
Putting the Pieces Together
The PictureBox control allows you to define hot regions inside displayed images. Each region is painted on a copy of the image—the map image—using a distinct color, possibly a color not used in the rest of the image. After you drop the PictureBox control onto a form, you set the map image and populate the hot spot collection. The hot spot collection informs the control about "hot" colors in the image. By default, when the user clicks on a hot region, the HotSpotClicked event is fired. Here’s a typical event handler:
Sub PictureBox1_HotSpotClicked(ByVal sender As Object, _ ByVal e As HotSpotEventArgs) _ Handles PictureBox1.HotSpotClicked MessageBox.Show(e.HotSpot.Description) End Sub
If the PictureBox is enabled to capture mouse movements, it registers an internal handler for the MouseMove event. After the event is fired, the control will present a tooltip. Title and text of the tooltip usually reflect the values set in the hot spot element. However, these parameters can be changed in the client handler. As mentioned, the tooltip can also be canceled:
RaiseEvent HotSpotFound(Me, args) If Not args.CancelTooltip Then m_tooltip.ToolTipTitle = args.HotSpot.Title m_tooltip.SetToolTip(Me, args.HotSpot.Description) Else HideTooltip() End If
Figure 7 shows the control in action and the standard tooltip it displays. The title of the tooltip is changed programmatically and the description of the hot region is also displayed on the form’s status strip:
Sub PictureBox1_HotSpotFound(ByVal sender As Object, _ ByVal e As HotSpotEventArgs) _ Handles PictureBox1.HotSpotFound Info.Text = e.HotSpot.Description e.HotSpot.Title = "EMEA" End Sub
Figure 7** Context-Sensitive Tooltips Displayed on a Map **
Data Binding Support
The built-in PictureBox control features a bindable property called Image. The addition of the HotSpots collection raises the need of a stronger form of data binding. For example, it would be great if the collection could be populated from a database, thus saving developers from editing source files if a color or a description changed at some time. You might want to add a DataSource property and map its contents to the HotSpots collection or perhaps transform the HotSpots collection into a read/write property that can be set programmatically as an object. If you opt for the classic DataSource property, you also need to define a few mapper properties to indicate which fields on the bound data source map to bindable properties of the hot spot elements. For example, you might want to have DataDescriptionField, DataTitleField, DataColorField, DataValueField properties to map fields on the data source to hot spot elements. For DataColorField, you are responsible for inventing a translation algorithm that generates a .NET Framework color type based on a string or a number stored in the bound data source. This could be done easily using the ColorConverter class in the System.Drawing namespace.
The PictureBox control’s bindable Image property is marked with the Bindable attribute and added to the DataBindings collection. You might want to extend this behavior to the MapImage property, too. To make this happen, you can add the Bindable attribute to the MapImage property in the source code of the control:
<Bindable(True)> _ Public Property MapImage() As Image ... End Property
An ASP.NET ImageMap Control
I discussed context-sensitive picture box controls in the context of a Windows Forms application; however, it’s easy to build a similar ASP.NET control starting from either the Image or ImageMap control.
In ASP.NET, the ImageMap control provides behavior similar to what I’ve described throughout this column and defines a number of HotSpot regions. When the user clicks a hot spot, the control either posts back or navigates to a specified URL. The ASP.NET framework comes with a number of predefined HotSpot objects including the CircleHotSpot, RectangleHotSpot, and PolygonHotSpot classes. Also, you can derive from the abstract HotSpot class to define your own custom hot spot object, which you may find handy in situations like the one I’ve described here.
Send your questions and comments for Dino to cutting@microsoft.com.
Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino at cutting@microsoft.com or join the blog at weblogs.asp.net/despos.