Skip to main content

8a - Automated Accessibility Testing in CI — Core Tests & CI Gate

Contractor: TBA

Main Contact: TBA

Start date: 1 October 2026

End data: 30 November 2026

Abstract​

This milestone ensures that every other milestone's work is protected from regression in future releases. By codifying the test protocols from Milestone 1a into automated CI checks, it turns accessibility from a per-milestone deliverable into a continuous quality property of the codebase, attributable to specific pull requests and fixable at the point a regression is introduced.

Milestone 8a depends directly on the test protocols and acceptance criteria established in Milestone 1a (including the agreed baseline) and on the critical admin user journeys identified there. It feeds into later milestones in the testing cluster.

Scope of Work​

Deliverable 1: Automated End-to-End Accessibility Tests for Critical User Journeys​

Implement automated end-to-end accessibility tests with axe-core checks for all critical admin user journeys identified in Milestone 1a, at a minimum:

  • Login
  • Article create and edit
  • Media management
  • List views with filters
  • Installer
  • Updater

In addition, the automated tests must validate that Guided Tour steps do not reference or rely on inaccessible interactions, ensuring that onboarding flows remain operable with keyboard and assistive technologies as the UI evolves.

The tests must:

  • Run reliably and repeatedly in CI
  • Be aligned with the test protocols, AT/browser matrix, and acceptance criteria from Milestone 1a
  • Be structured so that adding further journeys later (for example by the contractors of Milestones 8b and onward) does not require restructuring the existing test base

The proposal should be explicit about what tools and libraries will be used for automation, and about any areas where automation is not feasible due to technology limitations. For example, if a proposal does not include automating AT flows, it should list that as out of scope.

Deliverable 2: Accessibility CI Gate in Joomla's GitHub Actions Pipeline​

Integrate an accessibility CI gate into Joomla's GitHub Actions pipeline. Pull requests that introduce new WCAG 2.2 A/AA violations beyond the agreed Milestone 1a baseline must fail the gate.

Failure output must be:

  • Actionable, with error messages explaining what broke and how to fix it
  • Specific enough to identify the failing journey, the failing rule, and (where possible) the failing element or selector
  • Stable enough to avoid flakiness, with a documented policy for triaging intermittent failures

The gate must be configured against the agreed baseline from Milestone 1a, with a documented process for evolving that baseline as accessibility improves in later milestones.

Deliverable 3: Documentation and Milestone Report​

Maintain the programme documentation framework and produce the milestone completion report in the format agreed at kick-off. The documentation must also include:

  • A contributor-facing guide explaining the CI gate, how to interpret a failure, and how to update tests when behaviour intentionally changes
  • A maintainer-facing guide explaining how to evolve the baseline as later milestones land

References​

  1. RFP on OpenSourceMatters