Milestone 1a – Expert Audit and Baseline Metrics
Deliverable 1: WCAG 2.2 and ATAG Audit
-
Web Content Accessibility Guidelines (WCAG) Report
Alternative formats/exports:
- PDF Snapshot
- JSON Export for import into the WCAG-EM Report Tool
- CSV exports: General, Jira Import, GitLab Import, List of Attachments
-
Authoring Tools Accessibility Guidelines (ATAG) Report
Alternative formats/exports:
- PDF Snapshot
- CSV exports: General, Jira Import, GitLab Import, List of Attachments
Deliverable 2: Baseline Metrics
The baseline metrics have been collected using a mix of automatic (axe) and user testing. These metrics can be used in subsequent milestones to track progress and its velocity. Find the overview of the results in the following table and the details further down on the page:
| Journey | axe errors |
User 1 | User 2 | User 3 | User 4 |
|---|---|---|---|---|---|
| Journey 1 | 3 | Friction | Completed | Completed | Completed |
| Journey 2 | 8 | Blocked | Blocked | Blocked | Friction |
| Journey 3 | 7 | Blocked | Friction | Friction | Friction |
| Journey 4 | 10 | Not tested | Completed | Friction | Friction |
| Journey 5 | 8 | Friction | Completed | Friction | Completed |
| Journey 6 | 3 | Workaround | Not tested | Blocked | Completed |
| AUS score | ∅ | 35 | 73 | 28 | 50 |
= Lower is better / = Higher is better / All metrics are based on the “blog” example installation.
Details of the Baseline Metrics deliverables
Automated Testing
For the axe scores above, the Accessibility Checker is used. See the Guide and readme file for more details.
User Testing
In addition to the numbers from the automated testing, four users tested the user journeys. Each journey was rated on a scale of completion. Afterwards the users answered questions to determine the AUS (Accessibility Usability Scale) score.
The Accessible Usability Scale (AUS) is a free tool to measure the usability of a digital product for assistive technology users. Inspired by the System Usability Scale (SUS), the Accessible Usability Scale consists of ten questions administered at the end of a user experience to calculate a score.
The scale for completion for every journey is:
- Completed independently – The user completed the task without intervention.
- Completed with friction – The user completed the task, but there was significant friction to complete it.
- Completed with workaround – The user completed the task, but they did not take the optimal way to do it.
- Blocked or Failed – The task was not completed.
- Not Tested – The task was not attempted (usually because of time constraints).
Find detailed guidance and results here:
- User Testing Guide – How to set up user testing, how to facilitate it, and how to interpret results. (Google Doc Original)
- Results:
- Overview (in Google Sheets)
- User 1 (in Google Sheets)
- User 2 (in Google Sheets)
- User 3 (in Google Sheets)
- User 4 (in Google Sheets)
- Summary (in Google Sheets)
To repeat the user testing, see this template manuscript (in Google Docs) , and the results spreadsheet template (Zipped HTML export) (as Open Document Spreadsheet, in Google Sheets).
Deliverable 3: Accessibility Test Protocols and Acceptance Criteria
To ensure that testing can be integrated and re-used, these test protocols have been produced, including acceptance criteria for common components.
Deliverable 4: Publication Approach
This is this page.
Deliverable 5: Documentation & Reporting
Reports are delivered to OSM in a separate format.