Chapter 1: Introducing Hilo

The first Hilo sample application—the Hilo Browser—implements a touch-enabled user interface for browsing and selecting photos and images. You can download the code for the Hilo Browser application from the MSDN Code Gallery. This article introduces you to the Hilo Browser application and to developing applications for Windows 7 by describing some of the tools and APIs that you can use. In subsequent articles, we’ll drill into the details of the Hilo Browser application.

Using Hilo

The Hilo applications allow you to select, annotate and share collections of photos in a natural fashion. To do this, Hilo takes advantage of some of the key features in Windows 7: Libraries, Touch, Windows Ribbon, Direct2D, and Windows Animation.

Tablet computers enable you to interact with applications through a touch screen and to input data through a stylus. Hilo is designed to take advantage of both the touch and pen features while still being usable through computers that only have a mouse. The animation features of Windows 7 are used in Hilo to good effect, giving you a more natural paradigm to browse files and navigate. This design was chosen so as to move away from the table-based paradigm used by most Windows applications (for example, the Tree View and List View controls), and instead to present data in a format that facilitates the use of the Touch interface while making data access simpler and more natural.

The central Hilo application is the Hilo Browser as shown in Figure 1. The top half of the browser window is the carousel. Figure 1 shows that the Pictures folder has six subfolders and these are shown on the same orbital in the carousel. Through Touch you can rotate these folders around the orbital to bring the folder of your choice to the front. In this view the carousel will only show one complete orbital that you can spin, however, the carousel will also show partial orbitals representing the parent folders. For example, above the single, complete, orbital in Figure 1 you can see the partial orbital of the Pictures folder.

Figure 1 The Hilo Browser showing the carousel and the media pane

The lower half of the Hilo Browser window is the media pane which shows the images in the selected folder. The media pane shows the first few images, and other images are accessible by touching (or with a mouse, clicking) the media pane arrows.

The media pane is designed for a different paradigm: touch sensitive screens. Visually, the location of a finger touch on the screen is quite imprecise, so rather than adding a small scroll bar to the media pane, in the Hilo Browser, the media pane is the scroll bar. The scroll bar is redundant when you can drag the items themselves using touch.

So in the Hilo Browser you have two ways to display additional items: you can either touch the scroll arrows on the left and right of the pane, or you can flick anywhere in the media pane in the direction that you want the image list to move. This is a very natural gesture, and illustrates how having a touch interface means that application designers have to change how they think users will interact with user interface (UI) items. While gestures are very natural for people accessing the UI through touch, such gestures are unnatural with a mouse, so mouse users will use the scroll arrows or the mouse scroll wheel. The Hilo Browser also allows you to scroll images using the Page Up and Page Down keys, illustrating how successful applications can provide input features for mouse and keyboard users as well as Touch users.

Similarly, you will find that the rotating carousel is a natural action to perform with your finger on a touch screen. We are used to moving cards or photographs around a table, and the carousel provides a similar paradigm for accessing folders. Mouse users have full access to the carousel and they too can enjoy spinning the folders around an orbital.

When you navigate through the folder structure, the current selected folder is placed on the stack in the top left. Figure 1 shows that initially, Pictures is at the top of the stack. When you select a subfolder from the carousel, for example Summer, this new folder is placed on the top of the stack, as shown in Figure 2. As you continue through the folder structure each parent folder is placed on the stack. The arrow beneath the stack allows you to move up a level.

Figure 2 The Hilo Browser showing the stack in the top left corner

You can expand the stack at any time through the Touch interface or by clicking the mouse. When you expand the stack all the folders are shown (Figure 3) in a breadcrumb trail showing the folders in the stack. To the right of the breadcrumb trail is the grayed out orbital of the subfolders at the bottom of the trail.

Figure 3 The Hilo Browser showing the stack expanded

The orbital of each parent folder is shown as part of the expanded stack, but only the folders that are in the breadcrumb trail are shown. Furthermore, you cannot rotate the carousel orbitals in this view. However, you can select any folder in the breadcrumb trail which gives you a quick way to move to another folder and display its subfolders in the carousel. Restricting the breadcrumb trail like this to show only the folder in the trail gives a clean and less cluttered interface than, for example, the Tree View used in Windows Explorer, and makes it much easier for you to focus on the relevant folders.

To revert back to the previous carousel view you merely have to touch or click on a folder. In this way you can expand and collapse the stack, making navigation through the folder tree a simple and natural action.

Developing Applications for Windows 7

A commercial application has to be functional and (typically) provides services that are not offered by other products. However, merely functional is not enough in a competitive market. Applications have to offer a compelling user experience and this where the features of Windows 7 become very important. The Hilo Browser takes full advantage of these features. There are four main technologies used in the Hilo Browser: Direct2D, Windows Animation, the Touch API, and the Shell API.

Creating a User Interface with Direct2D

An important aspect of creating a compelling application is to provide a UI that is both visually appealing and practical. Modern computers have graphics cards with powerful graphics processing units (GPU) and use high resolution monitors. Direct2D is written for modern GPUs and provides access to the full capabilities of the GPUs. A Direct2D application, on a fully featured graphics card, will use hardware acceleration. However, to allow as many graphics cards as possible to be programmed with Direct2D, the underlying library checks the capabilities of the card’s GPU, and if the GPU does not support a feature the DirectX® application programming interface will provide a software implementation. This means that developers write the same code regardless of the graphics cards used.

Direct2D uses a different paradigm from Graphics Device Interface (GDI) programming in previous versions of Windows. With GDI programming, the program code determines how the pixels on the screen will change, and the computer’s central processing unit (CPU) performs the calculations to change those pixels. In Direct2D programming, the program indicates how the display will change, and the library tells the graphics card’s GPU to do the required calculations and update the display. This frees up the computer’s CPU to perform data crunching work, and results in a faster, more responsive application. Furthermore, GPUs support advanced features like antialiasing and alpha channel (opacity) blending that allows you to provide stunning user interfaces.

The Hilo Browser uses these features of Direct2D to full advantage. For example, the orbitals shown in the carousel (Figure 3) are drawn with a gradient brush.

Using Windows Animation

The Animation Manager enables developers to animate sequences of UI elements. In its simplest form, animation is a series of frames where one or more items change. The change may be the size, the color, the position or orientation of the UI item, and the change itself may be smooth or abrupt, linear or logarithmic. Constructing the series of frames into a storyboard to be played back has historically required a lot of coding and a lot of CPU processing at runtime. Furthermore, to make the animation as smooth as possible, the code must ensure that frames are displayed synchronized with the refresh rate of the monitor.

The Hilo Browser uses animation to good effect. Some animation is functional, for example the carousel displays folders by using a rotating paradigm that scales the size of the folder icon and caption according to the perceived position of the folder on the carousel. This means that the folder nearest to the user (the folder that the user is interested in) is shown larger with a larger caption than the other folders.

In other cases animation is meant to provide a compelling user experience. For example, when you touch a folder you zoom into the carousel, which means that the previous orbital expands until it reaches the existing orbitals shown at the top of the screen. The selected folder also moves to the top of the stack. At the same time the media pane is populated with thumbnails of as many images as the pane will display. The Hilo Browser displays these images, sliding in one by one from the left, as if they are playing cards being dealt on a table. This animation is more than just making the screen update more interesting; it serves an important purpose too. The human brain is more sensitive to movement than to static images, so this animation brings the user’s attention to the thumbnails being shown. As a developer you have to draw the user’s attention to the important data items, and the Hilo Browser shows how you can do this with animation.

Using the Touch API

The natural behavior is for users to select and move items on the screen by pointing to them with their finger or stylus. The Touch API allows you to provide this behavior to your applications, improving the user experience.

In the Hilo Browser, the user spins the carousel by dragging a folder with their finger and, as would be the case in real life, when the user removes their finger from the screen the carousel continues to spin but decelerates to a stop. The user can stop the carousel while it is spinning by touching a folder with their finger. By building in such natural interactions with the application, the designers of the Hilo Browser make it easy for a first time user to learn.

Using the Shell API

The Hilo Browser is a visually rich application. Standard folders are displayed using the well-known folder icon. User-created folders are displayed showing a preview of the folder contents. The media pane shows thumbnails of the images in the selected folder, and each thumbnail is the same size, regardless of the size of the actual image.

All of these features are provided by the Shell API. This API allows you to access the shell’s visual representation of items, such as folder images, as well as the icon or thumbnail for individual files The Shell, and Windows Explorer display these images, and your application can do this too, which means that your application displays are consistent with the Windows 7 user experience.

The Hilo Browser displays a thumbnail for the images in a folder. There are many image types and so the Hilo Browser must be able to extract the image information and create a thumbnail in a bitmap format that can be displayed on screen. To do this, the Hilo Browser uses the Shell API to obtain the image information from a file, and it then uses the Windows Imaging Component (WIC) to convert the image into a bitmap that can be displayed with the Direct2D API. WIC has support to load and manipulate images for all the standard image formats (TIFF, GIF, JPG, PNG, ICO, BMP) and metafile formats.

Furthermore, because loading and processing images is time consuming, there is a possibility of making an application sluggish. This is not the case with the Hilo Browser. Partly this is due to the clever use of animation when the images are displayed in the media pane, but the main reason is that the Hilo Browser provides code that asynchronously loads images on a background thread. The Hilo Browser can do this because Direct2D is multi-threaded aware. As a consequence the Hilo Browser is responsive in spite of the image processing that it has to do.

Developing the Hilo Browser Application

The Hilo Browser application is the first in a series of sample applications that allow you to browse, select, and work with images. You can download the code for the Hilo Browser application from the MSDN Code Gallery.

The next articles in the series describe the design and implementation of the Hilo Browser application in detail. Before then though, it can be useful to review the high-level design decisions behind the Hilo Browser application. These decisions include the choice of development language, application framework, design paradigm, and tooling.

The rich user experience of Windows 7 is best accessed through a powerful, flexible language, and that means C++: by using C++ you can access the raw power of the APIs for Windows 7. To build the Hilo sample applications, all you need is Visual C++ Express and the Windows SDK for Windows 7, both of which are available as free downloads.

Hilo applications show how to design and develop an application for Windows 7. But while the code showcases the APIs for Windows 7, it is not wedded to any particular application framework. Instead, Hilo implement a lightweight common application layer that directly uses and highlights the APIs rather than obscuring them. This common application layer is used to support all of the Hilo applications. It illustrates the best practices for developing Windows applications, and while it is not complete—it was designed simply to provide the features needed by the Hilo applications—it does show the best practices used in designing re-usable frameworks and can be extended to provide additional features.

The key design decisions behind the code for the Hilo Browser are simplicity and readability. That is the main reason why the common application layer is lightweight, but it also means that error checking and tracing is kept to a minimum. This is not because errors and tracing are unimportant (in fact they are vital aspects of any code design) but because extensive use of the tracing will distract from the main aim of showing how to use the APIs.

The Hilo Browser is an object orientated C++ design using the best practices of interface programming, abstraction and object factory patterns. The design is also chosen to give a responsive user interface and to this end, uses asynchronous data handling through a background thread. You can use all of these classes in your own projects.

Lastly, application development requires a powerful and fully featured development environment. Visual Studio 2010 offers an integrated environment where you, as a developer, can use your existing skills to code and debug your projects. Visual Studio provides powerful tools to help you deliver quality code quickly. There are four editions delivering varying levels of tool support which means that you can pay for the level of tool support you need.

Conclusion

This article provided an introduction to the Hilo Browser application. It introduced the features of the Hilo Browser, as well as the key technologies for Windows 7 used to implement those features. The next article covers how to set up your development environment, and explains how to install Visual C++ and the Windows SDK for Windows 7 so that you can compile and run the Hilo Browser source code.