Short answer: A regression suite for a government web app is a fixed set of automated and manual accessibility checks that runs on every release, so fixes made for WCAG 2.1 Level AA stay fixed. Build it from your most used pages and complete processes, run automated checks on every change, and add scheduled manual checks for what tools cannot judge.
Sources: ADA.gov fact sheet; W3C WAI, Evaluating Web Accessibility Overview.
Why does a government web app need regression testing?
The ADA Title II web rule sets WCAG 2.1 Level AA as the technical standard. Governments serving 50,000 or more people must comply by April 26, 2027. Smaller governments and special district governments must comply by April 26, 2028.
The deadline is not the end. ADA.gov says that after the compliance date, governments must continue to make sure their web content and mobile apps meet WCAG 2.1 Level AA. A release that breaks a form label or a focus style puts a fixed page back out of conformance.
W3C advises teams to evaluate accessibility early and throughout development. A regression suite is how that advice becomes a routine.
Sources: ADA.gov fact sheet; W3C WAI, Evaluating Web Accessibility Overview.
What should the suite cover first?
Use the sampling logic in the W3C evaluation methodology. It asks for a structured sample that covers common views, essential functionality, and the technologies the product relies on. Common views are the views relevant to the entire product, usually reached from the header, navigation or footer.
It also says to include every view in a series that presents a complete process, such as a payment or a permit application. Record the default sequence and any commonly used branches that are critical to finishing the process.
For a typical agency app, start with these:
- The home page and the main navigation
- Search and search results
- Each complete process, from first step to confirmation page
- Login, error and timeout states
- Each shared template and component, such as header, footer, forms, tables and dialogs
Source: W3C, WCAG Evaluation Methodology (WCAG-EM) 2.0, steps 3.1 and 3.3.
What can automation catch, and what can it not?
W3C states that no tool alone can determine whether a site meets accessibility standards. Its guide on selecting tools says tools can help with manual review and can sometimes produce false or misleading results.
Section508.gov names 3 ways to validate conformance: automated, manual and hybrid. Use all 3 in a regression suite, each at a different point in the release cycle.
- Automated: runs on every pull request. It catches missing labels, missing alt attributes, contrast failures and invalid markup.
- Manual: runs before each release on the changed areas. A person checks keyboard use, focus order, and whether alt text is meaningful.
- Hybrid: a tool guides a person through checks it cannot automate.
Sources: W3C WAI, Selecting Web Accessibility Evaluation Tools; Section508.gov, Test for Accessibility.
Regression suite template
Copy this table into your test plan. Adjust the pages to your own app and keep the criteria. Each row is one test case with an owner and a trigger.
| Test case | WCAG 2.1 criterion | Method | When it runs |
|---|---|---|---|
| Every shared component has a name, role and value | 4.1.2 Name, Role, Value (A) | Automated, plus screen reader spot check | Every pull request; manual on component change |
| Form fields have labels and clear errors | 3.3.1, 3.3.2 (A) | Automated, then manual submit with errors | Every pull request; manual on form change |
| Text and controls meet contrast ratios | 1.4.3, 1.4.11 (AA) | Automated | Every pull request |
| Page works with keyboard only | 2.1.1 Keyboard (A) | Manual | Each release, on changed pages |
| Focus is always visible and ordered | 2.4.7 Focus Visible (AA) | Manual | Each release, on changed pages |
| Content reflows at 320 CSS pixels | 1.4.10 Reflow (AA) | Manual or visual test | Each release |
| Headings, lists and tables use real markup | 1.3.1 Info and Relationships (A) | Automated, plus manual review | Every pull request; manual on template change |
| Each complete process works start to finish | Several, by step | Manual with keyboard and screen reader | Each release, and after any process change |
| Random sample of other pages | All AA criteria | Manual | Quarterly |
Sources: W3C, WCAG 2.1; W3C, How to Meet WCAG (Quick Reference).
How do you keep the suite useful over time?
The W3C methodology adds a random sample of 10% of the structured sample, drawn from pages not already in it. With 80 structured samples, that is 8 random pages. If the random pages show new content types or new findings, the structured sample was not representative enough and needs more samples.
When you re-run an evaluation, W3C suggests keeping part of the earlier sample for comparison and replacing the rest, typically about half. That keeps results comparable and widens coverage at the same time.
Two habits help most:
- Turn every escaped defect into a test. If a bug reached production, add a case for it.
- Fix components, then pages. A test on a shared component protects every page that uses it.
Source: W3C, WCAG Evaluation Methodology (WCAG-EM) 2.0, steps 3.2 and 4.3 and the section on re-running an evaluation.
What counts as a pass?
Set the pass rule before you start. ADA.gov says a deviation may be acceptable if it is so minor that it would not change a person with a disability's access, and the government must be able to show this. Its example is a contrast ratio of 4.45:1 against the required 4.5:1. It also says governments cannot use this part of the rule to avoid trying to meet WCAG 2.1 Level AA.
For a regression suite, the safe rule is simple. Treat any failed criterion as a defect, log it, and let the agency decide on any minor deviation.
W3C also notes that conformance claims cannot be made for an entire site from a sample alone. A passing suite shows no known regressions in what you test. It is not a conformance statement.
Sources: ADA.gov fact sheet; W3C, WCAG-EM 2.0.
What this means for subcontractors
A prime that wins an agency web project needs a way to show that accessibility holds after each release. A regression suite with a clear test log gives that evidence. It also shows the agency what was tested and when.
Keep the log in a form the prime can hand over: test case, page, result, build number and date. Related reading: the 12-point WCAG 2.1 AA checklist, the ADA Title II deadlines explained, and how primes work with Konnex. Our capability statement lists what we cover.
Quick answers
How often should a regression suite run?
Run automated checks on every code change and manual checks on changed areas before each release. W3C advises evaluating accessibility early and throughout development.
Source: W3C WAI, Evaluating Web Accessibility Overview.
Can automated tools prove WCAG 2.1 AA conformance?
No. W3C states that no tool alone can determine whether a site meets accessibility standards, and that tools can give false or misleading results.
Source: W3C WAI, Selecting Web Accessibility Evaluation Tools.
Does the Title II rule apply after the deadline passes?
Yes. ADA.gov says governments must continue to make sure their web content and mobile apps meet WCAG 2.1 Level AA after the compliance date.
Source: ADA.gov fact sheet.
This is general information, not legal advice.
Sources
- ADA.gov, Fact Sheet: New Rule on the Accessibility of Web Content and Mobile Apps
- W3C, WCAG Evaluation Methodology (WCAG-EM) 2.0
- W3C WAI, Evaluating Web Accessibility Overview
- W3C WAI, Selecting Web Accessibility Evaluation Tools
- Section508.gov, Test for Accessibility
- W3C, Web Content Accessibility Guidelines (WCAG) 2.1
Questions about your own project? Contact Konnex.