The Microsoft MSAA API cannot provide mapping for all ARIA roles as it does not have defined roles and properties for all those defined by The ARIA specification, the Microsoft UI Automation API can. UI automation provides exposure of all WAI-ARIA roles and properties via the ControlType and AriaRole properties.
Testing UI Automation AriaRole support
Testing of the AriaRole properties support requires the Windows 7 operating system and the use of an updated inspect tool, currently only available as part of the Windows 7 SDK. Results from initial testing show that all possible ARIA role values are correctly exposed via the UI automation AriaRole property, but the stated mapping of the ARIA roles to UI automation ControlType is incomplete. I have also re-tested MSAA role mapping taking into account roles that Microsft documented as unsupported via MSAA:
- 43 out of 51 – Of the 51 WAI-ARIA role (excluding abstract roles) values that can be mapped to MSAA: Internet explorer 8.0.7100.0 maps 43 correctly.
- 32 out of 50 – Of the 50 WAI-ARIA role (excluding abstract roles) values that can be mapped to UI automation ControlType: Internet explorer 8.0.7100.0 maps 32 correctly.
- 59 out of 59 – Of the 59 WAI-ARIA role (excluding abstract roles) values that can be mapped via UI automation AriaRole property: Internet explorer 8.0.7100.0 maps 59 correctly.
Full Test Results: Comparison of ARIA roles exposed via MSAA and UI Automation in IE8
Practical Ramifications of these Results
In theory by using the UI Automation API, assistive technology can access all the ARIA role values used in web pages viewed using Internet Explorer 8 via the AriaRole property. Although this property can only be tested in Windows 7, the property itself should be available to Assistive Technology on Vista and XP.
I emailed Michael Curran, one of the developers of the NVDA screen reader, to pick his brains on UI automation, here is what he had to say:
Technically, the properties are available on the other operating systems, but there is no real easy way for an AT to actually get at UI Automation in general, unless running on Windows 7.
NVDA, and I’m sure many other ATs, do not make use of UI Automation in Internet Explorer, as we already access the DOM directly. And although we think its great that Microsoft is exposing this info via accessibility APIs now, the older way of getting at this info still works.
Also, I said that its pretty impossible for ATs to get at UI Automation in Vista / XP. This is true, but here is a little further detail:
- ATs written on the .net platform (since 2.0 I believe) have been able to access UI Automation through a managed API. However, very few ATs are written in .net, in particular I know of know screen readers that are.
- UIAutomationCore.dll was only introduced in Windows Vista. This is code that allows any normal win32 application to implement UI Automation in their GUI. This then means that although there may be some .net applications that expose UI Automation information running in XP, no native win32 application can expose UI Automation unless running on Windows Vista or above I believe. This does seem to mean then that I might have been wrong about saying that IE would expose UIA properties in XP, especially when IE is not a .net application I believe.
- Although the new unmanaged UI Automation client API is only available in Windows 7, it would be possible for an AT to inject “in process” and use UI Automation directly in applications such as Internet Explorer, in Windows Vista. Many ATs already inject in process to read web pages anyway, however as I previously noted, I’m not too sure on the uptake of this method as many ATs already have full access to the DOM.
As you can see, the whole UI Automation story is extremely complex, and I guess still not fully complete.
Thanks to Michael for providing such detailed information! The main point for me in relation to WAI-ARIA is that “many ATs already have full access to the DOM.” which suggests that as long as ARIA is being exposed correctly via the DOM and AT’s have access to the DOM, they can implement ARIA.
Comments
Many thanks for this Steve, it helps explain some of the difficulties I and others I know have been having trying to get ARIA to work with readers. Thanks
It should be noted that while many ATs use the DOM to create a flat representation of a document, they don’t necessarily use the DOM for interaction with widgets. This is certainly the case for NVDA’s IE support at present. ARIA widgets really should be exposed the same way as an equivalent, non-ARIA widget (i.e. by accessibility APIs), and interaction should therefore be the same. This is how Firefox and (to an extent) IE 8 handle ARIA.
You conclude:
“as long as ARIA is being exposed correctly via the DOM and AT’s have access to the DOM, they can implement ARIA.”
While it might be possible for an AT to handle everything (including interaction) concerning ARIA widgets via the DOM, this is not an elegant solution, is quite error prone and will result in widely varying support of ARIA in IE across different ATs.
Hi James, thanks for clarifying this.
So do you use MSAA to access information about ARIA widgets that are supported by MSAA? If so is MSAA used for IE and Firefox? How would you access (in NVDA) roles that are not supported by MSAA? landmark roles for example.
IN Firefox, we use MSAA/IAccessible2 to access information about all interactive ARIA widgets. We don’t use the ISimpleDOM* interfaces which provide DOM access. As you point out, ARIA roles which don’t map to MSAA/IA2 roles – noteably, landmarks – have to be treated differently. Firefox exposes an “xml-roles” object attribute to cover this, and we would need to parse this to handle landmarks. (Note that NVDA does not currently support ARIA landmarks.)
Our IE code is still heavily under development, so we haven’t done any specific work for ARIA in IE. I would say we will use MSAA to access ARIA widgets wherever possible, but we may need to resort to interrogating the DOM if IE’s mapping of ARIA to MSAA proves insufficient. UIA is also another option, but we would only be able to utilise that on Windows 7, which makes it somewhat problematic for users using earlier versions of Windows.
“…no native win32 application can expose UI Automation unless running on Windows Vista or above I believe.”
I believe the minimum requirements for UIA implementation is .NET framework 3.0, which comes with Vista as standard. However you could install the framework on xp and write UIA client / server providers.
“..UIA is also another option, but we would only be able to utilise that on Windows 7, which makes it somewhat problematic for users using earlier versions of Windows.”
Watch this space..