What's New in UI Automation?
Many new features were added to the Microsoft UI Automation API for Windows 8 and Windows 8.1. They fall into these categories:
- Support for Text-based Controls
- Provider Interfaces
- Client Interfaces
- Control Pattern Identifiers
- Control Pattern Availability Property Identifiers
- Control Pattern Property Identifiers
- Annotation Type Identifiers
- Text Attribute and Style Identifiers
- Support for Drag-and-Drop Elements
- Drag-and-Drop Interfaces
- Drag-and-Drop Control Pattern Identifiers
- Drag-and-Drop Control Pattern Availability Property Identifiers
- Drag-and-Drop Control Pattern Property Identifiers
- Drag-and-Drop Event Identifiers
- Support for the System Caret
- Support for Live Regions
- Support for Windowless ActiveX Controls
- Support for Handling Unresponsive Applications
- Provider Functions
- Miscellaneous Identifiers
- Miscellaneous Client APIs
- Support for the TextEdit Control Pattern
- Showing Context Menus
- Related topics
Here are the UI Automation API elements that are available starting with Windows 8 or Windows 8.1.
Support for Text-based Controls
For Windows 8, many API elements were added to improve UI Automation support for text-based controls. Providers can use the new API elements to expose rich information about the content of text-based controls, and clients can access the rich information and use it to create a better experience for users of accessibility tools.
Provider Interfaces
- IAnnotationProvider
- IObjectModelProvider
- ISpreadsheetProvider
- ISpreadsheetItemProvider
- IStylesProvider
- ITextChildProvider
- ITextProvider2
- ITransformProvider2
Client Interfaces
- IUIAutomationAnnotationPattern
- IUIAutomationObjectModelPattern
- IUIAutomationSpreadsheetPattern
- IUIAutomationSpreadsheetItemPattern
- IUIAutomationStylesPattern
- IUIAutomationTextChildPattern
- IUIAutomationTextPattern2
- IUIAutomationTransformPattern2
Control Pattern Identifiers
- UIA_AnnotationPatternId
- UIA_ObjectModelPatternId
- UIA_SpreadsheetPatternId
- UIA_SpreadsheetItemPatternId
- UIA_StylesPatternId
- UIA_TextChildPatternId
- UIA_TextPattern2Id
- UIA_TransformPattern2Id
Control Pattern Availability Property Identifiers
- UIA_IsAnnotationPatternAvailablePropertyId
- UIA_IsObjectModelPatternAvailablePropertyId
- UIA_IsSpreadsheetPatternAvailablePropertyId
- UIA_IsSpreadsheetItemPatternAvailablePropertyId
- UIA_IsStylesPatternAvailablePropertyId
- UIA_IsTextChildPatternAvailablePropertyId
- UIA_IsTextPattern2AvailablePropertyId
- UIA_IsTransformPattern2AvailablePropertyId
Control Pattern Property Identifiers
- UIA_AnnotationAnnotationTypeIdPropertyId
- UIA_AnnotationAnnotationTypeNamePropertyId
- UIA_AnnotationAuthorPropertyId
- UIA_AnnotationDateTimePropertyId
- UIA_AnnotationTargetPropertyId
- UIA_SpreadsheetItemFormulaPropertyId
- UIA_SpreadsheetItemAnnotationObjectsPropertyId
- UIA_SpreadsheetItemAnnotationTypesPropertyId
- UIA_StylesExtendedPropertiesPropertyId
- UIA_StylesFillColorPropertyId
- UIA_StylesFillPatternColorPropertyId
- UIA_StylesFillPatternStylePropertyId
- UIA_StylesShapePropertyId
- UIA_StylesStyleIdPropertyId
- UIA_StylesStyleNamePropertyId
- UIA_Transform2CanZoomPropertyId
- UIA_Transform2ZoomLevelPropertyId
- UIA_Transform2ZoomMaximumPropertyId
- UIA_Transform2ZoomMinimumPropertyId
Annotation Type Identifiers
- AnnotationType_Comment
- AnnotationType_Footer
- AnnotationType_FormulaError
- AnnotationType_GrammarError
- AnnotationType_Header
- AnnotationType_Footer
- AnnotationType_TrackChanges
- AnnotationType_Unknown
Text Attribute and Style Identifiers
- UIA_AnnotationObjectsAttributeId
- UIA_AnnotationTypesAttributeId
- UIA_CaretBidiModeAttributeId
- UIA_CaretPositionAttributeId
- UIA_LinkAttributeId
- UIA_StyleIdAttributeId
- UIA_StyleNameAttributeId
- StyleId_BulletedList and StyleId_NumberedList added to Style Identifiers (Windows 8.1)
For information about how to implement and use the new API elements that support text-based controls, see these topics:
- Annotation Control Pattern
- ObjectModel Control Pattern
- Spreadsheet Control Pattern
- SpreadsheetItem Control Pattern
- Styles Control Pattern
- Control-specific Object Model
- Retrieving Annotations
- Retrieving Visual Styles
- Accessing Spreadsheet Content
- Text and TextRange Control Patterns
Support for Drag-and-Drop Elements
These Windows 8 API elements expose information about elements that can be dragged as part of a drag-and-drop operation, and elements that can be the drop target of a drag-and-drop operation:
Drag-and-Drop Interfaces
Drag-and-Drop Control Pattern Identifiers
Drag-and-Drop Control Pattern Availability Property Identifiers
Drag-and-Drop Control Pattern Property Identifiers
- UIA_DragIsGrabbedPropertyId
- UIA_DragDropEffectPropertyId
- UIA_DragDropEffectsPropertyId
- UIA_DragGrabbedItemsPropertyId
- UIA_DropTargetDropTargetEffectPropertyId
- UIA_DropTargetDropTargetEffectsPropertyId
Drag-and-Drop Event Identifiers
- UIA_Drag_DragStartEventId
- UIA_Drag_DragCancelEventId
- UIA_Drag_DragCompleteEventId
- UIA_DropTarget_DragEnterEventId
- UIA_DropTarget_DragLeaveEventId
- UIA_DropTarget_DroppedEventId
Support for the System Caret
These Windows 8 API elements support exposing information about the system caret:
- CaretBidiMode enumeration
- CaretPosition enumeration
- ITextProvider2::GetCaretRange method
- IUIAutomationTextPattern2::GetCaretRange method
- UIA_CaretBidiModeAttributeId attribute identifier
- UIA_CaretPositionAttributeId attribute identifier
Support for Live Regions
These Windows 8 API elements support exposing information about Accessible Rich Internet Applications (ARIA) live regions:
- UIA_LiveRegionChangedEventId event identifier
- EVENT_OBJECT_LIVEREGIONCHANGED structure
- IUIAutomationElement2::CachedLiveSetting property
- IUIAutomationElement2::CurrentLiveSetting property
Support for Windowless ActiveX Controls
Windows 8 includes these new Windows Accessibility API interfaces that simplify the task of implementing accessibility for windowless Microsoft ActiveX controls:
- IAccessibleHostingElementProviders
- IAccessibleWindowlessSite
- IRawElementProviderHostingAccessibles
- IRawElementProviderWindowlessSite
For more information, see Windowless ActiveX Control Accessibility.
Support for Handling Unresponsive Applications
These Windows 8 APIs enable a UI Automation client application to better detect and respond to an unresponsive provider:
- IUIAutomation2::ConnectionTimeout property
- IUIAutomation2::TransactionTimeout property
Provider Functions
These UI Automation provider functions were added in Windows 8:
- UiaDisconnectAllProviders
- UiaDisconnectProvider
- UiaIAccessibleFromProvider
- UiaProviderForNonClient
- UiaProviderFromIAccessible
Miscellaneous Identifiers
- UIA_OptimizeForVisualContentPropertyId property identifier
- UIA_SemanticZoomControlTypeId control type identifier; see also SemanticZoom Control Type
- UIA_IsPeripheralPropertyId property identifier(Windows 8.1)
- UIA_AppBarControlTypeId control type identifier (Windows 8.1); see also AppBar Control Type
Miscellaneous Client APIs
These UI Automation client APIs were added in Windows 8:
- IUIAutomation2::AutoSetFocus method
- IUIAutomationElement2::CachedOptimizeForVisualContent property
- IUIAutomationElement2::CurrentOptimizeForVisualContent property
These UI Automation client APIs were added in Windows 8.1:
- IUIAutomation3::CachedIsPeripheral property
- IUIAutomation3::CurrentIsPeripheral property
Support for the TextEdit Control Pattern
The TextEdit control pattern is added in Windows 8.1. The pattern is related to the Text and TextRange patterns and supports the ability to expose programmatic text-edit actions to UI Automation.
Provider interface: ITextEditProvider
Client interface: IUIAutomationTextEditPattern
Control pattern identifier: UIA_TextEditPatternId
Pattern availability property identifier: UIA_IsTextEditPatternAvailablePropertyId
Supporting API: UiaRaiseTextEditTextChangedEvent provider function, TextEditChangeType enumeration.
Showing Context Menus
In Windows 8.1 methods are added that can invoke the context menus that are owned by an element, or that are associated with a text range.
For providers: ITextRangeProvider2::ShowContextMenu method and IRawElementProviderSimple2::ShowContextMenu method.
For clients: IUIAutomation3::ShowContextMenu method and IUIAutomationTextRange2::ShowContextMenu method.