Bewerken

Pen interactions and Windows Ink in Windows apps

  • Artikel

Hero image of the Surface Pen.
Surface Pen (available for purchase at the Microsoft Store).

Overview

Optimize your Windows app for pen input to provide both standard pointer device functionality and the best Windows Ink experience for your users.

Note

This topic focuses on the Windows Ink platform. For general pointer input handling (similar to mouse, touch, and touchpad), see Handle pointer input.

Using ink in your Windows app

Use Windows Pen and Ink to build more engaging enterprise apps

The Windows Ink platform, together with a pen device, provides a natural way to create digital handwritten notes, drawings, and annotations. The platform supports capturing digitizer input as ink data, generating ink data, managing ink data, rendering ink data as ink strokes on the output device, and converting ink to text through handwriting recognition.

In addition to capturing the basic position and movement of the pen as the user writes or draws, your app can also track and collect the varying amounts of pressure used throughout a stroke. This information, along with settings for pen tip shape, size, and rotation, ink color, and purpose (plain ink, erasing, highlighting, and selecting), enables you to provide user experiences that closely resemble writing or drawing on paper with a pen, pencil, or brush.

Note

Your app can also support ink input from other pointer-based devices, including touch digitizers and mouse devices. 

The ink platform is very flexible. It is designed to support various levels of functionality, depending on your requirements.

For Windows Ink UX guidelines, see Inking controls.

Components of the Windows Ink platform

Component Description
InkCanvas A XAML UI platform control that, by default, receives and displays all input from a pen as either an ink stroke or an erase stroke.
For more information about how to use the InkCanvas, see Recognize Windows Ink strokes as text and Store and retrieve Windows Ink stroke data.
InkPresenter A code-behind object, instantiated along with an InkCanvas control (exposed through the InkCanvas.InkPresenter property). This object provides all default inking functionality exposed by the InkCanvas, along with a comprehensive set of APIs for additional customization and personalization.
For more information about how to use the InkPresenter, see Recognize Windows Ink strokes as text and Store and retrieve Windows Ink stroke data.
InkToolbar A XAML UI platform control containing a customizable and extensible collection of buttons that activate ink-related features in an associated InkCanvas.
For more information about how to use the InkToolbar, see Add an InkToolbar to a Windows app inking app.
IInkD2DRenderer Enables the rendering of ink strokes onto the designated Direct2D device context of a Universal Windows app, instead of the default InkCanvas control. This enables full customization of the inking experience.
For more information, see the Complex ink sample.

Basic inking with InkCanvas

To add basic inking functionality, just place an InkCanvas UWP platform control on the appropriate page in your app.

By default, the InkCanvas supports ink input only from a pen. The input is either rendered as an ink stroke using default settings for color and thickness (a black ballpoint pen with a thickness of 2 pixels), or treated as a stroke eraser (when input is from an eraser tip or the pen tip modified with an erase button).

Note

If an eraser tip or button is not present, the InkCanvas can be configured to process input from the pen tip as an erase stroke.

In this example, an InkCanvas overlays a background image.

Note

An InkCanvas has default Height and Width properties of zero, unless it is the child of an element that automatically sizes its child elements, such as StackPanel or Grid controls.

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Grid.RowDefinitions>
        <RowDefinition Height="Auto"/>
        <RowDefinition Height="*"/>
    </Grid.RowDefinitions>
    <StackPanel x:Name="HeaderPanel" Orientation="Horizontal" Grid.Row="0">
        <TextBlock x:Name="Header"
                   Text="Basic ink sample"
                   Style="{ThemeResource HeaderTextBlockStyle}"
                   Margin="10,0,0,0" />            
    </StackPanel>
    <Grid Grid.Row="1">
        <Image Source="Assets\StoreLogo.png" />
        <InkCanvas x:Name="inkCanvas" />
    </Grid>
</Grid>

This series of images shows how pen input is rendered by this InkCanvas control.

Screenshot of the blank InkCanvas with a background image. Screenshot of the InkCanvas with ink strokes. Screenshot of the InkCanvas with one stroke erased.
The blank InkCanvas with a background image. The InkCanvas with ink strokes. The InkCanvas with one stroke erased (note how erase operates on an entire stroke, not a portion).

The inking functionality supported by the InkCanvas control is provided by a code-behind object called the InkPresenter.

For basic inking, you don't have to concern yourself with the InkPresenter. However, to customize and configure inking behavior on the InkCanvas, you must access its corresponding InkPresenter object.

Basic customization with InkPresenter

An InkPresenter object is instantiated with each InkCanvas control.

Note

The InkPresenter cannot be instantiated directly. Instead, it is accessed through the InkPresenter property of the InkCanvas. 

Along with providing all default inking behaviors of its corresponding InkCanvas control, the InkPresenter provides a comprehensive set of APIs for additional stroke customization and finer-grained management of the pen input (standard and modified). This includes stroke properties, supported input device types, and whether input is processed by the object or passed to the app for processing.

Note

Standard ink input (from either pen tip or eraser tip/button) is not modified with a secondary hardware affordance, such as a pen barrel button, right mouse button, or similar mechanism.

By default, ink is supported for pen input only. Here, we configure the InkPresenter to interpret input data from both pen and mouse as ink strokes. We also set some initial ink stroke attributes used for rendering strokes to the InkCanvas.

To enable mouse and touch inking, set the InputDeviceTypes property of the InkPresenter to the combination of CoreInputDeviceTypes values that you want.

public MainPage()
{
    this.InitializeComponent();

    // Set supported inking device types.
    inkCanvas.InkPresenter.InputDeviceTypes =
        Windows.UI.Core.CoreInputDeviceTypes.Mouse |
        Windows.UI.Core.CoreInputDeviceTypes.Pen;

    // Set initial ink stroke attributes.
    InkDrawingAttributes drawingAttributes = new InkDrawingAttributes();
    drawingAttributes.Color = Windows.UI.Colors.Black;
    drawingAttributes.IgnorePressure = false;
    drawingAttributes.FitToCurve = true;
    inkCanvas.InkPresenter.UpdateDefaultDrawingAttributes(drawingAttributes);
}

Ink stroke attributes can be set dynamically to accommodate user preferences or app requirements.

Here, we let a user choose from a list of ink colors.

<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Grid.RowDefinitions>
        <RowDefinition Height="Auto"/>
        <RowDefinition Height="*"/>
    </Grid.RowDefinitions>
    <StackPanel x:Name="HeaderPanel" Orientation="Horizontal" Grid.Row="0">
        <TextBlock x:Name="Header"
                   Text="Basic ink customization sample"
                   VerticalAlignment="Center"
                   Style="{ThemeResource HeaderTextBlockStyle}"
                   Margin="10,0,0,0" />
        <TextBlock Text="Color:"
                   Style="{StaticResource SubheaderTextBlockStyle}"
                   VerticalAlignment="Center"
                   Margin="50,0,10,0"/>
        <ComboBox x:Name="PenColor"
                  VerticalAlignment="Center"
                  SelectedIndex="0"
                  SelectionChanged="OnPenColorChanged">
            <ComboBoxItem Content="Black"/>
            <ComboBoxItem Content="Red"/>
        </ComboBox>
    </StackPanel>
    <Grid Grid.Row="1">
        <Image Source="Assets\StoreLogo.png" />
        <InkCanvas x:Name="inkCanvas" />
    </Grid>
</Grid>

We then handle changes to the selected color and update the ink stroke attributes accordingly.

// Update ink stroke color for new strokes.
private void OnPenColorChanged(object sender, SelectionChangedEventArgs e)
{
    if (inkCanvas != null)
    {
        InkDrawingAttributes drawingAttributes =
            inkCanvas.InkPresenter.CopyDefaultDrawingAttributes();

        string value = ((ComboBoxItem)PenColor.SelectedItem).Content.ToString();

        switch (value)
        {
            case "Black":
                drawingAttributes.Color = Windows.UI.Colors.Black;
                break;
            case "Red":
                drawingAttributes.Color = Windows.UI.Colors.Red;
                break;
            default:
                drawingAttributes.Color = Windows.UI.Colors.Black;
                break;
        };

        inkCanvas.InkPresenter.UpdateDefaultDrawingAttributes(drawingAttributes);
    }
}

These images shows how pen input is processed and customized by the InkPresenter.

Screenshot that shows the InkCanvas with default black ink strokes.

The InkCanvas with default black ink strokes.

Screenshot of the InkCanvas with user selected red ink strokes.

The InkCanvas with user selected red ink strokes.

To provide functionality beyond inking and erasing, such as stroke selection, your app must identify specific input for the InkPresenter to pass through unprocessed for handling by your app.

Pass-through input for advanced processing

By default, InkPresenter processes all input as either an ink stroke or an erase stroke, including input modified by a secondary hardware affordance such as a pen barrel button, a right mouse button, or similar. However, users typically expect some additional functionality or modified behavior with these secondary affordances.

In some cases, you might also need to expose additional functionality for pens without secondary affordances (functionality not usually associated with the pen tip), other input device types, or some type of modified behavior based on a user selection in your app's UI.

To support this, InkPresenter can be configured to leave specific input unprocessed. This unprocessed input is then passed through to your app for processing.

Example - Use unprocessed input to implement stroke selection

The Windows Ink platform does not provide built-in support for actions that require modified input, such as stroke selection. To support features like this, you must provide a custom solution in your apps.

The following code example (all code is in the MainPage.xaml and MainPage.xaml.cs files) steps through how to enable stroke selection when input is modified with a pen barrel button (or right mouse button).

  1. First, we set up the UI in MainPage.xaml.

    Here, we add a canvas (below the InkCanvas) to draw the selection stroke. Using a separate layer to draw the selection stroke leaves the InkCanvas and its content untouched.

    Screenshot of the blank InkCanvas with an underlying selection canvas.

      <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
    <Grid.RowDefinitions>
      <RowDefinition Height="Auto"/>
      <RowDefinition Height="*"/>
    </Grid.RowDefinitions>
    <StackPanel x:Name="HeaderPanel" Orientation="Horizontal" Grid.Row="0">
      <TextBlock x:Name="Header"
        Text="Advanced ink customization sample"
        VerticalAlignment="Center"
        Style="{ThemeResource HeaderTextBlockStyle}"
        Margin="10,0,0,0" />
    </StackPanel>
    <Grid Grid.Row="1">
      <!-- Canvas for displaying selection UI. -->
      <Canvas x:Name="selectionCanvas"/>
      <!-- Inking area -->
      <InkCanvas x:Name="inkCanvas"/>
    </Grid>
  </Grid>

  2. In MainPage.xaml.cs, we declare a couple of global variables for keeping references to aspects of the selection UI. Specifically, the selection lasso stroke and the bounding rectangle that highlights the selected strokes.

      // Stroke selection tool.
  private Polyline lasso;
  // Stroke selection area.
  private Rect boundingRect;

  3. Next, we configure the InkPresenter to interpret input data from both pen and mouse as ink strokes, and set some initial ink stroke attributes used for rendering strokes to the InkCanvas.

    Most importantly, we use the InputProcessingConfiguration property of the InkPresenter to indicate that any modified input should be processed by the app. Modified input is specified by assigning InputProcessingConfiguration.RightDragAction a value of InkInputRightDragAction.LeaveUnprocessed. When this value is set, the InkPresenter passes through to the InkUnprocessedInput class, a set of pointer events for you to handle.

    We assign listeners for the unprocessed PointerPressed, PointerMoved, and PointerReleased events passed through by the InkPresenter. All selection functionality is implemented in the handlers for these events.

    Finally, we assign listeners for the StrokeStarted and StrokesErased events of the InkPresenter. We use the handlers for these events to clean up the selection UI if a new stroke is started or an existing stroke is erased.

    Screenshot of the Advances ink customization sample app showing the inkcanvas with default black ink strokes.

      public MainPage()
  {
    this.InitializeComponent();

    // Set supported inking device types.
    inkCanvas.InkPresenter.InputDeviceTypes =
      Windows.UI.Core.CoreInputDeviceTypes.Mouse |
      Windows.UI.Core.CoreInputDeviceTypes.Pen;

    // Set initial ink stroke attributes.
    InkDrawingAttributes drawingAttributes = new InkDrawingAttributes();
    drawingAttributes.Color = Windows.UI.Colors.Black;
    drawingAttributes.IgnorePressure = false;
    drawingAttributes.FitToCurve = true;
    inkCanvas.InkPresenter.UpdateDefaultDrawingAttributes(drawingAttributes);

    // By default, the InkPresenter processes input modified by
    // a secondary affordance (pen barrel button, right mouse
    // button, or similar) as ink.
    // To pass through modified input to the app for custom processing
    // on the app UI thread instead of the background ink thread, set
    // InputProcessingConfiguration.RightDragAction to LeaveUnprocessed.
    inkCanvas.InkPresenter.InputProcessingConfiguration.RightDragAction =
        InkInputRightDragAction.LeaveUnprocessed;

    // Listen for unprocessed pointer events from modified input.
    // The input is used to provide selection functionality.
    inkCanvas.InkPresenter.UnprocessedInput.PointerPressed +=
        UnprocessedInput_PointerPressed;
    inkCanvas.InkPresenter.UnprocessedInput.PointerMoved +=
        UnprocessedInput_PointerMoved;
    inkCanvas.InkPresenter.UnprocessedInput.PointerReleased +=
        UnprocessedInput_PointerReleased;

    // Listen for new ink or erase strokes to clean up selection UI.
    inkCanvas.InkPresenter.StrokeInput.StrokeStarted +=
        StrokeInput_StrokeStarted;
    inkCanvas.InkPresenter.StrokesErased +=
        InkPresenter_StrokesErased;
  }

  4. We then define handlers for the unprocessed PointerPressed, PointerMoved, and PointerReleased events passed through by the InkPresenter.

    All selection functionality is implemented in these handlers, including the lasso stroke and the bounding rectangle.

    Screenshot of the selection lasso.

      // Handle unprocessed pointer events from modified input.
  // The input is used to provide selection functionality.
  // Selection UI is drawn on a canvas under the InkCanvas.
  private void UnprocessedInput_PointerPressed(
    InkUnprocessedInput sender, PointerEventArgs args)
  {
    // Initialize a selection lasso.
    lasso = new Polyline()
    {
        Stroke = new SolidColorBrush(Windows.UI.Colors.Blue),
        StrokeThickness = 1,
        StrokeDashArray = new DoubleCollection() { 5, 2 },
        };

        lasso.Points.Add(args.CurrentPoint.RawPosition);

        selectionCanvas.Children.Add(lasso);
    }

    private void UnprocessedInput_PointerMoved(
      InkUnprocessedInput sender, PointerEventArgs args)
    {
      // Add a point to the lasso Polyline object.
      lasso.Points.Add(args.CurrentPoint.RawPosition);
    }

    private void UnprocessedInput_PointerReleased(
      InkUnprocessedInput sender, PointerEventArgs args)
    {
      // Add the final point to the Polyline object and
      // select strokes within the lasso area.
      // Draw a bounding box on the selection canvas
      // around the selected ink strokes.
      lasso.Points.Add(args.CurrentPoint.RawPosition);

      boundingRect =
        inkCanvas.InkPresenter.StrokeContainer.SelectWithPolyLine(
          lasso.Points);

      DrawBoundingRect();
    }

  5. To conclude the PointerReleased event handler, we clear the selection layer of all content (the lasso stroke) and then draw a single bounding rectangle around the ink strokes encompassed by the lasso area.

    Screenshot of the selection bounding rect.

      // Draw a bounding rectangle, on the selection canvas, encompassing
  // all ink strokes within the lasso area.
  private void DrawBoundingRect()
  {
    // Clear all existing content from the selection canvas.
    selectionCanvas.Children.Clear();

    // Draw a bounding rectangle only if there are ink strokes
    // within the lasso area.
    if (!((boundingRect.Width == 0) ||
      (boundingRect.Height == 0) ||
      boundingRect.IsEmpty))
      {
        var rectangle = new Rectangle()
        {
          Stroke = new SolidColorBrush(Windows.UI.Colors.Blue),
            StrokeThickness = 1,
            StrokeDashArray = new DoubleCollection() { 5, 2 },
            Width = boundingRect.Width,
            Height = boundingRect.Height
        };

        Canvas.SetLeft(rectangle, boundingRect.X);
        Canvas.SetTop(rectangle, boundingRect.Y);

        selectionCanvas.Children.Add(rectangle);
      }
    }

  6. Finally, we define handlers for the StrokeStarted and StrokesErased InkPresenter events.

    These both just call the same cleanup function to clear the current selection whenever a new stroke is detected.

      // Handle new ink or erase strokes to clean up selection UI.
  private void StrokeInput_StrokeStarted(
    InkStrokeInput sender, Windows.UI.Core.PointerEventArgs args)
  {
    ClearSelection();
  }

  private void InkPresenter_StrokesErased(
    InkPresenter sender, InkStrokesErasedEventArgs args)
  {
    ClearSelection();
  }

  7. Here's the function to remove all selection UI from the selection canvas when a new stroke is started or an existing stroke is erased.

      // Clean up selection UI.
  private void ClearSelection()
  {
    var strokes = inkCanvas.InkPresenter.StrokeContainer.GetStrokes();
    foreach (var stroke in strokes)
    {
      stroke.Selected = false;
    }
    ClearDrawnBoundingRect();
   }

  private void ClearDrawnBoundingRect()
  {
    if (selectionCanvas.Children.Any())
    {
      selectionCanvas.Children.Clear();
      boundingRect = Rect.Empty;
    }
  }

Custom ink rendering

By default, ink input is processed on a low-latency background thread and rendered in-progress, or "wet", as it is drawn. When the stroke is completed (pen or finger lifted, or mouse button released), the stroke is processed on the UI thread and rendered "dry" to the InkCanvas layer (above the application content and replacing the wet ink).

You can override this default behavior and completely control the inking experience by "custom drying" the wet ink strokes. While the default behavior is typically sufficient for most applications, there are a few cases where custom drying might be required, these include:

  • More efficient management of large, or complex, collections of ink strokes
  • More efficient panning and zooming support on large ink canvases
  • Interleaving ink and other objects, such as shapes or text, while maintaining z-order
  • Drying and converting ink synchronously into a DirectX shape (for example, a straight line or shape rasterized and integrated into application content instead of as a separate InkCanvas layer).

Custom drying requires an IInkD2DRenderer object to manage the ink input and render it to the Direct2D device context of your Universal Windows app, instead of the default InkCanvas control.

By calling ActivateCustomDrying (before the InkCanvas is loaded), an app creates an InkSynchronizer object to customize how an ink stroke is rendered dry to a SurfaceImageSource or VirtualSurfaceImageSource.

Both SurfaceImageSource and VirtualSurfaceImageSource provide a DirectX shared surface for your app to draw into and compose into your application's content, although VSIS provides a virtual surface that’s larger than the screen for performant panning and zooming. Because visual updates to these surfaces are synchronized with the XAML UI thread, when ink is rendered to either, the wet ink can be removed from the InkCanvas simultaneously.

You can also custom dry ink to a SwapChainPanel, but synchronization with the UI thread is not guaranteed and there might be a delay between when the ink is rendered to your SwapChainPanel and when ink is removed from the InkCanvas.

For a full example of this functionality, see the Complex ink sample.

Note

Custom drying and the InkToolbar
If your app overrides the default ink rendering behavior of the InkPresenter with a custom drying implementation, the rendered ink strokes are no longer available to the InkToolbar and the built-in erase commands of the InkToolbar do not work as expected. To provide erase functionality, you must handle all pointer events, perform hit-testing on each stroke, and override the built-in "Erase all ink" command.

Topic Description
Recognize ink strokes Convert ink strokes to text using handwriting recognition, or to shapes using custom recognition.
Store and retrieve ink strokes Store ink stroke data in a Graphics Interchange Format (GIF) file using embedded Ink Serialized Format (ISF) metadata.
Add an InkToolbar to a Windows inking app Add a default InkToolbar to a Windows app inking app, add a custom pen button to the InkToolbar, and bind the custom pen button to a custom pen definition.

APIs

Samples

Archive Samples