August 2015
Volume 30 Number 8
Windows 10 - Modern Drag and Drop for Windows Universal Applications
This article is based on the public preview of Windows 10 and Visual Studio 2015.
Drag and drop is an intuitive way to transfer data within an application or between applications on the Windows desktop. It made its debut in Windows 3.1 with the File Manager and was then extended to all applications supporting OLE 2, such as Microsoft Office. When Windows 8 was released, Microsoft introduced a new type of Windows application, called a Windows Store app, which was designed for tablets and always displayed in full-screen mode. Because only one application was visible at any given time, cross-application drag and drop didn’t make sense and other ways of sharing data, such as the Share Charm, were developed. In Windows 10, however, applications on a desktop PC are once again running in windowed mode, which means multiple windows are displayed on the screen, and therefore drag and drop makes a come-back as a Universal Windows app API, with the addition of new features that enhance the UX.
Drag-and-Drop Concepts
Drag and drop lets the user transfer data between applications or within an application using a standard gesture (press-hold-and-pan with the finger or press-and-pan with a mouse or a stylus).
The drag source, which is the application or area where the drag gesture is triggered, provides the data to be transferred by filling a data package object that can contain standard data formats, including text, RTF, HTML, bitmaps, storage items or custom data formats. The source also indicates the kind of operations it supports: copy, move or link. When the pointer is released, drop occurs. The drop target, which is the application or area underneath the pointer, processes the data package and returns the type of operation it performed.
During drag and drop, the drag UI provides a visual indication of the type of drag-and-drop operation that’s taking place. This visual feedback is initially provided by the source but can be changed by the targets as the pointer moves over them.
Modern drag and drop is available on desktop, tablet and phone. It allows data transfer between or within any kind of application, including Classic Windows apps, although this article focuses on the XAML API for modern drag and drop.
Implementing Drag and Drop
The drag source and drop target play different roles. An application can have UI components that are only drag sources, only drop targets or both, such as the sample Photo Booth application shown in Figure 1.
.jpg "Drag Sources and Drop Targets")
Figure 1 Drag Sources and Drop Targets
A drag-and-drop operation is driven entirely by user input, so its implementation is almost exclusively event-based, as shown in Figure 2.
.png "Drag and Drop Events")
Figure 2 Drag and Drop Events
Implementing a Drag Source In Windows 8.1, a ListView can be the source of an in-app drag-and-drop operation if its CanDragItems property is set to true:
<ListView CanDragItems="True"
DragItemsStarting="ListView_DragItemsStarting"
ItemsSource="{Binding Pictures}"
ItemTemplate="{StaticResource PictureTemplate}"
/>
The application can handle the DragItemsStarting event on the source. This is still supported in Windows 10 with the addition of a DragItemsCompleted event, which wasn’t needed in Windows 8.1 applications where target and source must belong to the same process.
The main drag source for modern drag and drop is the UIElement, which gives access to all modern drag-and-drop features and is the main focus of this article.
One way to make UIElement draggable is to set its CanDrag property to true. This can be done in the markup or in the codebehind. The XAML framework handles gesture recognition and fires the DragStarting event to indicate the start of a drag operation. The application must configure the DataPackage by filling its content and indicating which operations are supported. The source application can put different formats in the DataPackage, which will make it compatible with more targets, as shown in Figure 3. The supported operations are defined in the DataPackageOperation type and can be Copy, Move, Link or any combination of these.
Figure 3 Handling DragStarting and Filling the DataPackage
private void DropGrid_DragStarting(UIElement sender,
DragStartingEventArgs args)
{
if (Picture == null)
{
args.Cancel = true;
}
else if (_fileSource != null)
{
args.Data.RequestedOperation =
DataPackageOperation.Copy | DataPackageOperation.Move;
args.Data.SetStorageItems(new IStorageItem[] { _fileSource });
...
}
You can cancel the Drag operation in the event handler by setting the Cancel property of the DragStartingEventArgs parameter; for example, in our sample application, an image placeholder will start a Drag operation only if it has received an image or a file.
The DragStartingEvent handler is also the place where the source application can customize the drag UI, which is explained later in this article.
In some cases, the application might want to use a special gesture to start a drag-and-drop operation, or want to allow dragging a control whose normal interactions interfere with the drag-and-drop gesture, for example, the TextBox, which already reacts to pointer-down events by changing its selection. In these cases, the application can implement its own gesture detection and then initiate a drag-and-drop operation by calling the StartDragAsync method. Note that this method expects a pointer identifier and, therefore, you can’t start a drag-and-drop operation with non-standard devices, such as a Kinect sensor. Once StartDragAsync is called, the rest of the drag and drop operation follows the same pattern as if CanDrag=True had been used, including the DragStarting event.
Once the user has released the pointer, the drag-and-drop operation is complete and the source is notified through the DropCompleted event, which contains the DataPackageOperation returned by the target on which the user released the pointer, or DataPackageOperation.None if the pointer was released on a target that doesn’t accept the data or if Cancel was pressed:
private void DropGrid_DropCompleted(UIElement sender, DropCompletedEventArgs args)
{
if (args.DropResult == DataPackageOperation.Move)
{
// Move means that we should clear our own image
Picture = null;
_bitmapSource = null;
_fileSource = null;
}
}
StartDragAsync returns an IAsyncOperation<DataPackageOperation>; the source application can handle the end of the operation either by waiting on the IAsyncOperation or by handling the DropCompleted event. Programmatically canceling after the DragStarting event is possible through the IAsyncOperation interface, but might be disturbing for the user.
Note that although both the ListView drag and drop and the UIElement drag and drop are implemented on the same system services and are fully compatible, they don’t raise the same events on the source side. That is, if a ListView has its CanDragItems property set to true, only DragItemsStarting and DragItemsCompleted are raised. DragStarting and DropCompleted are events related to the UIElement’s CanDrag property.
Implementing a Drop Target Any UIElement can be a drop target provided its AllowDrop property is set to true. During a drag-and-drop operation, the following events can be raised on a target: DragEnter, DragOver, DragLeave and Drop. These events already exist in Windows 8.1 but the DragEventArgs class has been extended in Windows 10 to give applications access to all the features of modern drag and drop. When handling a drag-and-drop event, the target application should first inspect the content of the DataPackage through the DataView property of the event argument; in most cases, checking for the presence of a data type is enough and it can be done synchronously. In some cases, such as with files, the application might have to check the type of the available files before accepting or ignoring the DataPackage. This is an asynchronous operation and requires the target application to take a deferral and later complete it (this pattern is detailed later in this article).
Once the target has determined if it can process the data, it must set the AcceptedOperation property of the DragEventArgs instance to allow the system to provide the right feedback to the user.
Note that if the application returns DataTransferOperation.None—or an operation not accepted by the source—from an event handler, drop won’t take place even if the user releases the pointer over the target; the DragLeave event will be raised instead.
The application can handle either DragEnter or DragOver; the AcceptedOperation returned by DragEnter is kept if DragOver isn’t handled. As DragEnter is called only once, it should be preferred to DragOver for performance reasons. However, in the case of nested targets, it’s necessary to return the correct value from DragOver in case a parent target might override it (setting Handled to true prevents the event from bubbling up to the parent). In the sample application, each photo placeholder checks for images in the DataPackage and routes the event to the parent grid only if no image is available, which allows the grid to accept Text even if it’s physically dropped on a placeholder (see Figure 4).
Figure 4 Handling DragEnter and Inspecting DataPackage
private async void DropGrid_DragEnter(object sender, DragEventArgs e)
{
if (!App.IsSource(e.DataView))
{
bool forceMove = ((e.Modifiers & DragDropModifiers.Shift) ==
DragDropModifiers.Shift);
if (e.DataView.Contains(StandardDataFormats.Bitmap))
{
_acceptData = true;
e.AcceptedOperation = (forceMove ? DataPackageOperation.Move :
DataPackageOperation.Copy);
e.DragUIOverride.Caption = "Drop the image to show it in this area";
e.Handled = true;
}
else if (e.DataView.Contains(StandardDataFormats.StorageItems))
{
// Notify XAML that the end of DropGrid_Enter does
// not mean that we have finished to handle the event
var def = e.GetDeferral();
_acceptData = false;
e.AcceptedOperation = DataPackageOperation.None;
var items = await e.DataView.GetStorageItemsAsync();
foreach (var item in items)
{
try
{
StorageFile file = item as StorageFile;
if ((file != null) && file.ContentType.StartsWith("image/"))
{
_acceptData = true;
e.AcceptedOperation = (forceMove ? DataPackageOperation.Move :
DataPackageOperation.Copy);
e.DragUIOverride.Caption = "Drop the image to show it in this area";
break;
}
}
catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
e.Handled = true;
// Notify XAML that now we are done
def.Complete();
}
}
// Else we let the event bubble on a possible parent target
}
Advanced Concepts
Customizing the visual feedback The only feedback that OLE 2 drag and drop provided was to change the mouse cursor according to the target’s answer to the DragOver event. Modern drag and drop allows more advanced scenarios as the visual feedback provided to the user is richer. The drag UI consists of three parts: visual content, glyph and caption.
The visual content represents the data being dragged. It might be the dragged UIElement (if the source is a XAML application); a standard icon chosen by the system based on the DataPackage’s content; or a custom image set by the application.
The glyph reflects the type of operation accepted by the target. It can take four different shapes corresponding to the values of the DataPackageOperation type. It can’t be customized by applications, but it can be hidden.
The caption is a description given by the target. Depending on the target application, a Copy operation might, for example, be the addition of a song to a playlist, the upload of a file to OneDrive or a plain file copy. The caption allows more precise feedback than the glyph and plays a role very similar to a tooltip.
The table in Figure 5 shows how the source and target can customize these different parts. When the pointer isn’t over a drop target, the drag UI is exactly what the source has configured. When the pointer is over a drop target, some parts of the visual might be overridden by the target; all overrides are cleared when the pointer leaves the target.
Figure 5 Customizations Available to the Source and Target
| Source | Target | |
| Visual Content | Default = dragged element Can use system-generated content based on DataPackage Can use any bitmap |
Default = what has been set by source Can’t use system-generated content Can use any bitmap Can show or hide |
| Glyph | No access | Aspect based on AcceptedOperation Can show or hide |
| Caption | No access | Can use any string Can show or hide |
When a drag-and-drop operation starts, if the source application doesn’t try to customize the drag UI in the DragStarting event handler, a snapshot of the dragged UIElement is taken by XAML and used as the drag UI content. The initial UIElement is still displayed in its original position, which is a different behavior from ListView, where the dragged ListViewItems are hidden from their initial position. Because the snapshot of the dragged UIElement is taken after the DragStarting event has been raised, it’s possible to trigger a visual state change during this event to alter the snapshot. (Note that the UIElement’s state is also altered and, even if it’s restored, a light flicker might happen.)
When handling a DragStarting event, the drag source can customize the visual feedback through the DragUI property of the DragStartingEventArgs class. For example, asking the system to use the DataPackage’s content to generate the visual content is done through SetContentFromDataPackage, as shown in Figure 6.
Figure 6 Using SetContentFromDataPackage to Generate Visual Content
private void DropGrid_DragStarting(UIElement sender, DragStartingEventArgs args)
{
...
if (_fileSource != null)
{
args.Data.RequestedOperation = DataPackageOperation.Copy | DataPackageOperation.Move;
args.Data.SetStorageItems(new IStorageItem[] { _fileSource });
args.DragUI.SetContentFromDataPackage();
}
else if (_bitmapSource != null)
{
args.Data.RequestedOperation = DataPackageOperation.Copy | DataPackageOperation.Move;
args.Data.SetBitmap(_bitmapSource);
args.DragUI.SetContentFromDataPackage();
}
}
You can set a custom bitmap as the content of the drag UI using two different classes: the well-known XAML BitmapImage class or a new Windows 10 class, SoftwareBitmap. If this bitmap is a resource of the application, it’s easier to use a BitmapImage and initialize it with the URI of the resource:
private void SampleBorder_DragStarting(UIElement sender, DragStartingEventArgs args)
{
args.Data.SetText(SourceTextBox.SelectedText);
args.DragUI.SetContentFromBitmapImage(new BitmapImage(new Uri(
"ms-appx:///Assets/cube.png", UriKind.RelativeOrAbsolute)));
}
If the bitmap must be generated on the fly when the drag operation starts or when the pointer enters a drop target, a SoftwareBitmap can be created from the buffer generated by the XAML RenderTargetBitmap class, which generates a bitmap containing the visual representation of a UIElement, as shown in Figure 7. This UIElement must be in the XAML visual tree but doesn’t need to be on the visible part of the page. Because RenderTargetBitmap does this rendering asynchronously, it’s necessary to take a deferral here so XAML knows if the bitmap isn’t ready when the event handler is finished and waits for the deferral to be completed to update the content of the Drag UI. (We’ll explain the deferral mechanism in more detail in the next section of this article.)
Figure 7 Customizing Drag UI Content with RenderTargetBitmap and SoftwareBitmap
private async void PhotoStripGrid_DragStarting(UIElement sender, DragStartingEventArgs args)
{
if ((Picture1.Picture == null) || (Picture2.Picture == null)
|| (Picture3.Picture == null) || (Picture4.Picture == null))
{
// Photo Montage is not ready
args.Cancel = true;
}
else
{
args.Data.RequestedOperation = DataPackageOperation.Copy;
args.Data.SetDataProvider(StandardDataFormats.Bitmap, ProvideContentAsBitmap);
App.SetSource(args.Data);
var deferral = args.GetDeferral();
var rtb = new RenderTargetBitmap();
const int width = 200;
int height = (int)(.5 + PhotoStripGrid.ActualHeight / PhotoStripGrid.ActualWidth
* (double)width);
await rtb.RenderAsync(PhotoStripGrid, width, height);
var buffer = await rtb.GetPixelsAsync();
var bitmap = SoftwareBitmap.CreateCopyFromBuffer(buffer, BitmapPixelFormat.Bgra8, width, height,
BitmapAlphaMode.Premultiplied);
args.DragUI.SetContentFromSoftwareBitmap(bitmap);
deferral.Complete();
}
}
Of course, if the SoftwareBitmap has already been generated—and it could be cached for subsequent drag-and-drop operations—no deferral is needed.
For both SetContentFromBitmapImage and SetContentFromSoftwareBitmap, you can specify an anchor point that indicates how to position the drag UI relative to the pointer position. If you use the overload without anchor point parameter, the top-left corner of the custom bitmap will follow the pointer. The DragStartingEventArgs GetPosition method returns the position of the pointer relative to any UIElement, which can be used to set the start position of the drag UI exactly where the dragged UIElement is located.
On the target side, the different parts of the dragged visual can be customized either in DragEnter or DragOver event handlers. Customization is made through the DragUIOverride property of DragEventArgs class, which exposes four SetContentFrom methods identical to DragUI on the source side, as well as four properties that let you hide different parts of the DragUI and change the caption. Finally, DragUIOverride also exposes a Clear method that resets all overrides of the DragUI made by the target.
Asynchronous Operations The Windows Universal Applications API enforces an asynchronous pattern for all operations that can take more than a few milliseconds. This is particularly critical in the case of drag and drop as such operations are completely driven by the user. Because of its richness, drag and drop uses three different asynchronous patterns: asynchronous calls, deferrals and callbacks.
Asynchronous calls are used when the application calls a system API that might take some time to complete. This pattern is now well-known by Windows developers and is made simple by the async and await keywords in C# (or create_task and then in C++). All methods that retrieve data from DataPackage follow this pattern, such as GetBitmapAsync, which our sample application uses to retrieve an image stream reference, as shown in Figure 8.
Figure 8 Using Asynchronous Calls to Read the DataPackage
private async void DropGrid_Drop(object sender, DragEventArgs e)
{
if (!App.IsSource(e.DataView))
{
bool forceMove = ((e.Modifiers & DragDropModifiers.Shift) ==
DragDropModifiers.Shift);
if (e.DataView.Contains(StandardDataFormats.Bitmap))
{
// We need to take a deferral as reading the data is asynchronous
var def = e.GetDeferral();
// Get the data
_bitmapSource = await e.DataView.GetBitmapAsync();
var imageStream = await _bitmapSource.OpenReadAsync();
var bitmapImage = new BitmapImage();
await bitmapImage.SetSourceAsync(imageStream);
// Display it
Picture = bitmapImage;
// Notify the source
e.AcceptedOperation = forceMove ? DataPackageOperation.Move :
DataPackageOperation.Copy;
e.Handled = true;
def.Complete();
}
...
}
Deferrals are used when the XAML framework calls back into an application’s code that might itself issue an asynchronous call before returning a value awaited by the framework. This pattern was not widely used in previous versions of XAML, so let’s take the time to analyze it. When a button generates a Click event, it’s a one-way call in the sense that no value needs to be returned by the application. If an asynchronous call is made by the application, its result will be available after the completion of the Click event handler, but this is perfectly fine because this handler doesn’t return a value.
On the other hand, when XAML raises a DragEnter or a DragOver event, it expects the application to set the AcceptedOperation property of the event arguments to indicate whether the content of the DataPackage can be handled. If the application only cares about the available data type inside the DataPackage, this can still be done synchronously, for example:
private void DropTextBox_DragOver(object sender, DragEventArgs e)
{
bool hasText = e.DataView.Contains(StandardDataFormats.Text);
e.AcceptedOperation = hasText ? DataPackageOperation.Copy :
DataPackageOperation.None;
}
However, if, for example, the application can accept only some file types, it must not only check the data types inside the DataPackage, it must access the data, as well, which can only be done asynchronously. This means that the execution of the code is suspended until the data has been read and that the DragEnter (or DragOver) event handler will be completed before the application knows whether it can accept the data. This scenario is exactly the purpose of the deferral: by getting a deferral from the DragEventArg object, the application tells XAML that it will defer some of its processing, and by completing the deferral, the application notifies XAML that this processing is done and the output properties of the DragEventArgs instance have been set. Refer back to Figure 4 to see how our sample application checks for StorageItems after having got a deferral.
The deferral can also be used when the customization of the drag UI content on the target side requires asynchronous operations such as the RenderTargetBitmap’s RenderAsync method.
On the source side of a drag-and-drop operation, DragStartingEventArgs exposes a deferral, too, whose purpose is to allow the operation to start as soon as the event handler terminates (even if the deferral hasn’t been completed) in order to provide the fastest feedback to the user even if creating a bitmap to customize the Drag UI takes some time.
Callbacks are used in the DataPackage to defer supplying the data until it’s really needed. With this mechanism, the source application can advertise several formats in the DataPackage, but only the data actually read by a target will have to be prepared and transferred. In many cases, the callback will never be called—for example, if no target can understand the corresponding data format—which is a nice performance optimization.
Note that, in many cases, providing the real data will require an asynchronous call and, therefore, the DataProviderRequest parameter of this callback exposes a deferral so applications can notify that they need more time to provide the data and then that the data is available, as shown in Figure 9.
Figure 9 Deferring the Data Until It Is Actually Read
private void DeferredData_DragStarting(UIElement sender,
DragStartingEventArgs args)
{
args.Data.SetDataProvider(StandardDataFormats.Text, ProvideDeferredText);
}
async void ProvideDeferredText(DataProviderRequest request)
{
var deferral = request.GetDeferral();
var file = await KnownFolders.DocumentsLibrary.GetFileAsync(fileName);
var content = await FileIO.ReadTextAsync(file);
request.SetData(content);
deferral.Complete();
}
Wrapping Up
When writing an application that manipulates standard data formats such as files, images or text, you should consider implementing drag and drop as this is an operation both natural and well-known to users. The basics of drag and drop are already familiar to those who have programmed with Windows Forms and Windows Presentation Foundation, which reduces the learning curve of this rich feature, with its specific concepts such as drag UI customization and the relatively unused patterns such as the deferral pattern. If you just want to support basic drag-and-drop scenarios, you can rely on your previous experience and have a straightforward implementation or, if you prefer, you can fully exploit the new features to provide a tailored experience to the user.
Anna Pai is a software engineer on the Xbox Team. She previously worked on Silverlight, Silverlight for Windows Phone and then XAML for Windows and Windows Phone.
Alain Zanchetta is a software engineer on the Windows XAML Team. Previously he was an architect in the consulting division of Microsoft France.
Thanks to the following Microsoft technical expert for reviewing this article: Clément Fauchère
Clement Fauchere is software engineer in Windows Shell Team at Microsoft.