Accessibility checker

Built on Gherkin, Playwright and axe-core.

Reusable framework for accessibility audits along real user journeys. Journeys are written in plain-English Gherkin, executed with Playwright, scanned with axe-core, and exported to CSV.

What the Accessibility Checker does

The app lets a QA enter the base URL and credentials (stored locally in the gitignored webapp-settings.json), select which user journeys to run, create or edit journeys in the browser, follow the log live, cancel an accessibility check that is still going, and download the CSV. The visualizer is linked from the header, so saved results can be compared without running anything. The server binds to localhost only.

Starting and stopping the Accessibility Checker

The first start downloads everything that is needed: a private Node.js (only if the machine has none), the dependencies, and the test browser. That takes a few minutes and needs an internet connection. Every later start skips all of that and just opens the Accessibility Checker in your browser at http://localhost:4700.

Mac

  1. Double-click Start Accessibility Checker (mac).command.
  2. First launch only: macOS may block the file because it was downloaded. Right-click it and choose Open. If there is no Open choice, go to System Settings, then Privacy & Security, and press “Open Anyway”.
  3. To stop the server, double-click Stop Accessibility Checker (mac).command.

Or from Terminal, paste these lines (adjust the first path to where the folder is):

cd "$HOME/Downloads/Accessibility Checker"
./"Start Accessibility Checker (mac).command"
cd "$HOME/Downloads/Accessibility Checker"
./"Stop Accessibility Checker (mac).command"

Windows

  1. Double-click Start Accessibility Checker (windows).bat.
  2. First launch only: if SmartScreen warns about an unrecognised app, press “More info” and then “Run anyway”.
  3. To stop the server, double-click Stop Accessibility Checker (windows).bat.

Or from Command Prompt, paste these lines (adjust the first path to where the folder is):

cd /d "%USERPROFILE%\Downloads\Accessibility Checker"
"Start Accessibility Checker (windows).bat"
cd /d "%USERPROFILE%\Downloads\Accessibility Checker"
"Stop Accessibility Checker (windows).bat"

Linux

Open a terminal and paste these lines (adjust the first path to where the folder is). Running through bash works even when unzipping dropped the file’s execute permission:

cd "$HOME/Downloads/Accessibility Checker"
bash "Start Accessibility Checker (linux).sh"
cd "$HOME/Downloads/Accessibility Checker"
bash "Stop Accessibility Checker (linux).sh"

If the test browser asks for missing system libraries on the first check, the start script prints the exact sudo command to run.

Without the start and stop files (any system)

The launcher files are only a convenience. From a terminal, the setup and start are three commands. Node.js version 18 or newer must be installed first: the installer from https://nodejs.org also installs the npm command that everything below uses.

One-time setup, run inside the folder:

npm install
npx playwright install chromium

Start the server. After the one-time setup above, this single command is all that is ever needed:

npm start

Then open http://localhost:4700 in a browser. The server runs in the terminal window; stop it by pressing Ctrl+C there, or by closing the window. Unlike the start files, nothing keeps running in the background afterwards.

If the terminal answers “command not found: npm”, this machine has no Node.js on its search path. Install Node.js from https://nodejs.org and open a new terminal window, or skip the terminal route entirely: the start files above find or download Node.js on their own.

Accessibility visualizer

The Accessibility visualizer (axe-visualizer.html) is a fully self-contained page (no server needed; double-click it, or open /visualizer from the web app). Load one or more result CSV files to get:

Severity is shown as a shape as well as a colour (octagon, triangle, diamond, circle) so the scale still reads for colour-blind viewers.

Each loaded accessibility check can be removed again with its Remove button, which is the quickest way to change which checks a comparison covers. Everything stays in the browser; nothing is uploaded. Multiple CSVs of the same accessibility check are deduplicated automatically.

Running from the command line

Running journeys without the web app reads its settings from a .env file instead: copy .env.example to .env and fill in the base URL and credentials first. 

npm test               # run all journeys (headless)
npx cucumber-js --tags @j2       # run only features with a given tag

After an accessibility check you get:

Writing a user journey

Create a .feature file in features/. Available steps:

Given I am logged in to the Joomla admin panel
When I log in on the frontend
Given I go to "/administrator/index.php?option=com_content"
When I reload the page
When I go back
When I wait for 2 seconds
When I wait for the text "Article saved"

Interacting (by accessible name: doubles as a labelling check)

When I click the "Save" button
When I click the "Articles" link
When I click the "Content" menu item
When I click on "Some visible text"
When I fill in "Title" with "My test article"
When I select "Published" in the "Status" dropdown
When I check "Remember me"
When I press the "Escape" key
When I click the element "#toolbar-new button"   # CSS escape hatch
When I accept all cookies if a cookie banner appears
When I search for "Joomla"                        # site search + Enter
Then I should see the text "Article saved"

Joomla-specific

When I type "Body text" in the rich text editor        # TinyMCE
When I switch to the "System" tab
When I check row 1 in the list                          # admin list tables
When I click on "System Links" in the dialog            # iframe modals
When I click the "Select" button in the dialog
When I insert the first image from the media dialog

Scanning

# Audit mode: records issues for the report, never fails the journey:
Then I scan the page for accessibility issues
Then I scan the "Article list" page for accessibility issues

# Gate mode: fails the scenario if the page has violations:
Then the page should have no accessibility violations
Then the "Dashboard" page should have no accessibility violations

Scan as often as you like mid-journey: after opening a modal, after saving, in error states. Every scan becomes rows in the same report.

Configuration (.env)

Variable Purpose
BASE_URL Site under test, e.g. https://staging.example.com
JOOMLA_ADMIN_USERNAME / JOOMLA_ADMIN_PASSWORD Admin login
HTTP_AUTH_USERNAME / HTTP_AUTH_PASSWORD Server password (.htaccess), if the site sits behind one. Each run first asks the site whether it wants one; if it does not, these are ignored, so they can be left in place when a site stops using a server password
AXE_TAGS Which axe rule sets to run (default WCAG 2.2 AA)

Google Sheets

There is no built-in Sheets export. The CSV imports cleanly into Google Sheets via File, then Import, whenever a spreadsheet is needed.

CSV columns

ID, Run, Feature, Scenario, Steps taken, Page, URL, Axe rule, Impact, WCAG tags, Issue, Description, How to fix, Selector, HTML snippet, Screenshot, Help URL.

ID numbers the issues within a run, so a run date plus an ID names exactly one finding, and names its screenshot. Screenshots are always taken: one per scanned page and one close-up per issue with the violating element highlighted, saved as reports/screenshots/<check>/<ID>-<issue>-<page>.png (numbered when the same issue occurs more than once on a page). Steps taken is the Gherkin the journey ran to reach the page that was scanned, so a finding can be reproduced by hand.

Notes