| Kevin Babbitt | 805b781 | 2021-06-14 18:18:16 | [diff] [blame] | 1 | # UI Automation |
| 2 | |
| 3 | [UI Automation (UIA)](https://docs.microsoft.com/en-us/windows/win32/winauto/entry-uiauto-win32) |
| Greg Thompson | 59edafe2 | 2023-06-15 08:05:16 | [diff] [blame] | 4 | is the modern accessibility API on Windows. The Chromium UIA provider is |
| 5 | currently under development. It can be enabled via the |
| 6 | `--enable-features=UiaProvider` browser command line switch. |
| Kevin Babbitt | 805b781 | 2021-06-14 18:18:16 | [diff] [blame] | 7 | |
| 8 | ## Key Features |
| 9 | |
| 10 | ### Clients and Providers |
| 11 | |
| 12 | UI Automation exposes two different sets of interfaces. One is intended for |
| 13 | clients such as assistive technologies and automation frameworks. The other is |
| 14 | intended for providers such as UI widget frameworks and applications that render |
| 15 | their own content. Chromium implements the UI Automation provider APIs. |
| 16 | |
| 17 | Clients and providers do not talk directly to one another. Instead, the |
| 18 | operating system gathers data from providers to present a unified tree view |
| 19 | across all open applications to the client. |
| 20 | |
| 21 | Further reading: |
| 22 | |
| 23 | * [UI Automation Provider Programmer's Guide](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-providerportal) |
| 24 | * [UI Automation Client Programmer's Guide](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-clientportal) |
| 25 | |
| 26 | ### Views of the UI Automation tree |
| 27 | |
| 28 | Clients have the ability to filter the UI Automation tree to various subsets of |
| 29 | nodes. Tools such as Windows Narrator take advantage of this capability to skip |
| 30 | over structural details that might be interesting to an automation framework but |
| 31 | aren't relevant to a screen reader. |
| 32 | |
| 33 | Providers can set two properties on a node to determine which views it appears |
| 34 | in: IsControlElement and IsContentElement. Getting the value of these properties |
| 35 | right is critical to ensuring assistive technologies can get the information |
| 36 | they need. Despite the names, there are many cases where nodes that might not be |
| 37 | considered a "control" should appear in the Control view of the tree. A good |
| 38 | litmus test is: if there's a reason a screen reader might be interested in a |
| 39 | node, it should appear in the control view. |
| 40 | |
| 41 | Further reading: |
| 42 | |
| 43 | * [UI Automation Tree Overview](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-treeoverview) |
| 44 | |
| 45 | ### TextPattern |
| 46 | |
| 47 | In addition to the tree view, UI Automation exposes a linear reading view |
| 48 | through an API known as TextPattern. This API allows a screen reader to navigate |
| 49 | text in more natural ways - characters, words, sentences, paragraphs, pages - |
| 50 | without worrying about the underlying tree structure. Windows Narrator relies |
| 51 | heavily on TextPattern for reading. |
| 52 | |
| 53 | Further reading: |
| 54 | |
| 55 | * [Text and TextRange Control Patterns](https://docs.microsoft.com/en-us/windows/win32/winauto/uiauto-implementingtextandtextrange) |
