Automated accessibility testing with cypress-axe

Accessibility is a crucial aspect of modern web development. It ensures that digital products are accessible to all users, including people with disabilities. Automated testing helps to identify potential issues early on and ensure that accessibility is considered from the outset. In this blog post, I'll show how to implement automated accessibility testing with cypress-axe. We will also take a look at relevant technologies such as the Cypress XRay for Jira plugin and tagging tests with cypress grep. In addition, you will learn how these tests can be integrated into CI/CD pipelines such as Jenkins or GitLab and what role test data management plays in this. The importance of manual testing and how it can be complemented by automated procedures will also be discussed.

Legal background

In the public sector, the Barrier-free Information Technology Ordinance (BITV) 2.0 has been in place for some time. This ordinance sets out the guidelines from the WCAG 2.1 AA classification. Now, as a result of the Accessibility Strengthening Act (BFSG), it also applies to manufacturers, importers, dealers and service providers. It mainly concerns digital products and services such as e-commerce, banking services, e-books, audiovisual media and electronic communication. The WCAG 2.1 AA classification has also been set for these. However, there is a risk of fines from 28 June 2025 if this is not implemented.

Requirements for the lightweight approach

Step 1: Test automation with cypress-axe

• Executable pipelines in CI/CD systems such as Jenkins/Gitlab/Bitbucket/Github Actions
• Node installed
• Cypress installed
• Typescript selected

Step 2: Feedback on XRay Testmanagement for Jira

• Jira installed
• Technical Jira user with write permission for the project
• XRay Testmanagement for Jira installed

Introduction to Cypress and axe-core: a strong duo

Cypress is a popular end-to-end testing framework that impresses with its easy installation and usability. In combination with the axe-core library, automated accessibility tests can be seamlessly integrated into existing test procedures. axe-core is an open-source library that was developed specifically for analysing the accessibility of websites. By integrating it with Cypress, you can quickly and efficiently create accessibility tests that are carried out during your normal test runs.

Here's a real-world example: Imagine you're developing an e-commerce website. With Cypress-axe, you can ensure that all the important elements, such as navigation menus, forms, and buttons, are accessible. You write automated tests that check these components for accessibility. For example, a test could check whether all images have alternative texts and whether forms are correctly labelled.

Installation

1. Install Cypress-axe (for Cypress >= Version 10)

	
	npm install –save-dev cypress-axe
	

2. build in commands

	
	import ‘cypress-axe’
		npm install –save-dev cypress-axe
	

in der cypress/support/e2e.js

3. Adjust Tsconfig (when using Typescript in Cypress)

	
	"types": ["cypress", "cypress-axe"]
	

Add types to cypress-axe

First well maintainable navigation tests including accessibility

For the practical example, I decided on a fictitious website. The company P-A38-42 offers smart home and smart gardening solutions. In a first approach, the menu items in the left menu should be clicked automatically.

Create JSON file for URL

For the example, we have a nested menu that is structured as follows:

• Products

  • Smart Gardening

    • Irrigation Systems
  • Smart Home

    • Complete Systems
    • Energy Measurement
• Services

  • Training
  • Installation

The file is stored in the cpyress subfolder fixtures with the name navigation.json. This allows Cypress to access it directly without a data structure description.

Note: It would look even better as a YAML file instead of a JSON file, but here you still need the js-yaml parser plugin.

Each of these elements leads to a subpage and has a data-cy HTML inline element. This ensures that the individual items are clearly labelled and can be accessed independently of the actual IDs and CSS classes in Cypress.

Creating a page object

To ensure good maintainability, we use the Page Object Model Pattern from the outset. This is a design pattern in the field of test automation that serves to improve the structure and maintainability of test scripts.

In this pattern, each web page or page of an application is modelled as a class that defines the elements of the page as properties (buttons or text fields) and the actions that can be performed on them as methods (login or search).

Advantages

• Reusability: Once defined, page objects can be used in multiple tests.
• Maintainability: Changes to the user interface only need to be made in the corresponding page object class, instead of in all tests.
• Clarity: Test cases are easier to read and better structured because the logic of page interaction is separated from the test logic.

To do this, we create a folder pageobjects directly in the cypress directory. Here we create a file called topmenue.ts and fill it with the following content.

In the fixtures folder, we now create a JSON file called page-menue.json with the following content:

	
	{
		  "produkte": {
		 "selector": "[data-cy=produkte]",
		    "text": "Produkte",
		    "url": "/produkte",
		    "children": {
		      "smartGardening": {
		        "selector": "[data-cy=smart-gardening]",
		        "text": "Smart Gardening",
		        "url": "/produkte/smart-gardening",
		        "children": {
		          "bewaesserungssysteme": {
		            "selector": "[data-cy=bewaesserungssysteme]",
		            "text": "Bewässerungssysteme",
		            "url": "/produkte/smart-gardening/bewaesserungssysteme"
		          }
		        }
		      },
		      "smartHome": {
		        "selector": "[data-cy=smart-home]",
		        "text": "Smart Home",
		        "url": "/produkte/smart-home",
		        "children": {
		          "komplettsysteme": {
		            "selector": "[data-cy=komplettsysteme]",
		            "text": "Komplettsysteme",
		            "url": "/produkte/smart-home/komplettsysteme"
		          },
		          "energiemessung": {
		            "selector": "[data-cy=energiemessung]",
		            "text": "Energiemessung",
		            "url": "/produkte/smart-home/energiemessung"
		          }        
		 }
		      }
		    }
		  },
		  "dienstleistungen": {
		    "selector": "[data-cy=dienstleistungen]",
		    "text": "Dienstleistungen",
		    "url": "/dienstleistungen",
		    "children": {
		      "schulungen": {
		        "selector": "[data-cy=schulungen]",
		        "text": "Schulungen",
		        "url": "/dienstleistungen/schulungen"
		      },
		      "einbau": {
		        "selector": "[data-cy=einbau]",
		        "text": "Einbau",
		        "url": "/dienstleistungen/einbau"
		      }
		    }
		  }
		 }
	

Create Cypress Test

	
	import Page from '../support/pageObjects/topmenue'
		// Structure for menue items including submenues for recursive using
		interface PageData {
		  selector: string
		  text: string
		  url: string
		  children?: Record<string, PageData>
		}
		describe('Accessibility Tests', () => {
		  const page = new Page()
		  // Load pages from the fixture before each test
		  beforeEach(function () {
		    // Load fixture data
		    cy.fixture('menue-pages').then((pages) => {
		      this.pagesData = pages
		    })
		    // Inject axe for accessibility testing before each test
		    cy.injectAxe()
		  })
		  // Recursive function to test accessibility for a page and its children
		  function testPageAccessibility(pageData: PageData) {
		    it(`should check accessibility for `, function () {
		      // Visit the page
		      page.visit(pageData.url)
		      // Check accessibility with a specific scope or ignoring specific rules
		      cy.checkA11()
		    // Recursively test children if they exist
		    if (pageData.children) {
		      Object.values(pageData.children).forEach(child => {
		        testPageAccessibility(child)
		      })
		    }
		  }
		  // Iterate over each page in the fixture and run the accessibility test
		  it('should run accessibility tests on all pages', function () {
		    const pages: Record<string, PageData> = this.pagesData
		    Object.values(pages).forEach(pageData => {
		      testPageAccessibility(pageData)
		    })
		  })
		  // Optional: Add mobile tests
		  ['iphone-6', 'iphone-12'].forEach(viewport => {
		    it(`should check accessibility for all pages on  viewport`, function () {
		      cy.viewport(viewport)
		      const pages: Record<string, PageData> = this.pagesData
		      Object.values(pages).forEach(pageData => {
		        testPageAccessibility(pageData)
		      })
		    })
		  })
		})
	

Explanation of the code

This code performs accessibility tests for multiple pages of an application that are defined in a fixture file. It checks the accessibility of the pages and their sub-pages, and also runs the tests on different viewports (such as iPhone-6 and iPhone-12).

Importing the Page Object Model (POM) class

	
	import Page from '../support/pageObjects/page'
	

This import pulls in the Page Object Model (POM) class from another file.

Definition of the PageData interface

	
	interface PageData {
		  selector: string
		  text: string
		  url: string
		  children?: Record<string, PageData>
		}
	

A type definition for the structure of the menu items is created here. The PageData interface describes the elements of the pages that are tested, including their selector, the text to be displayed, the URL and optionally the children. This structure allows recursion to map hierarchical menu structures with submenus (children).

Initialising the test suite

	
	describe('Accessibility Tests', () => {
		  const page = new Page()
	

describe is the block that defines the test suite. All accessibility tests are grouped within this block. The constant page is an instance of the Page class, which is used for page views and possibly other interactions.

beforeEach hook to prepare the tests

	
	beforeEach(function () {
		  cy.fixture('menue-pages').then((pages) => {
		    this.pagesData = pages
		  })
		  cy.injectAxe()
		})
	

The beforeEach hook is executed before each test. Two things are prepared here:

• Loading data: cy.fixture() loads the menu pages from a JSON file and stores them in this.pagesData. This data structure contains the pages to be tested.
• Inserting the accessibility test tool: cy.injectAxe() is used to load the axe-core tool into the page, which is used to check accessibility.

Recursive function for accessibility tests

	
	function testPageAccessibility(pageData: PageData) {
		  it(`should check accessibility for `, function () {
		    page.visit(pageData.url)
		    cy.checkA11()
	

The testPageAccessibility function performs accessibility tests for each page and its sub-pages.

• The it block describes the individual test for the current page.
• Visit page: page.visit(pageData.url) visits the page to be tested.
• Accessibility check: cy.checkA11() checks the accessibility of the page.

Recursive sub-page checking

	
	if (pageData.children) {
		  Object.values(pageData.children).forEach(child => {
		    testPageAccessibility(child)
		  })
		}
	

If the page has sub-pages (children), this function calls the testPageAccessibility function recursively for each sub-page. This ensures that nested pages are also tested for accessibility.

Performing accessibility tests for all pages

	
	it('should run accessibility tests on all pages', function () {
		  const pages: Record<string, PageData> = this.pagesData
		  Object.values(pages).forEach(pageData => {
		    testPageAccessibility(pageData)
		  })
		})
	

Here, a test is created that iterates over all pages defined in the fixture file. For each page, the testPageAccessibility function is called, which performs the accessibility test. This ensures that all pages are tested.

Viewport tests for mobile devices

	
	['iphone-8', 'iphone-12'].forEach(viewport => {
		  it(`should check accessibility for all pages on  viewport`, function () {
		    cy.viewport(viewport)
		    const pages: Record<string, PageData> = this.pagesData    Object.values(pages).forEach(pageData => {
		      testPageAccessibility(pageData)
		    })
		  })
		})
	

This block runs additional tests on different viewports (iPhone-6 and iPhone-12). cy.viewport() is used to simulate the screen size, ensuring that pages render correctly and are accessible on different device sizes.

The same recursive function, testPageAccessibility, is called for each viewport to ensure that accessibility rules are followed for both mobile and desktop views.

Optimisation with Cypress XRay for Jira Plugin

The Cypress XRay for Jira Plugin extends the functionality of Cypress by enabling direct integration with Jira. XRay is a test management tool that integrates with Jira and allows for centralised management of test cases and specifications. With the plugin, you can link accessibility tests directly to user stories in Jira, which greatly improves the traceability and management of tests.

For example, a user story in Jira describes the implementation of a new feature on your website. With the XRay plugin, you can document and manage specific accessibility tests associated with that feature directly in Jira. This ensures that each feature is tested for accessibility from the outset and that the results of the tests are incorporated directly into the development process.

Test tagging with cypress grep and parallel execution

Cypress grep is a useful tool for filtering and running specific tests based on tags. This is particularly helpful if you want to isolate and run accessibility tests in a targeted manner. You can use tags like @a11y (short for accessibility) to mark accessibility tests and ensure that they are run independently of other test categories.

A typical use case would be to run accessibility tests as part of a CI/CD pipeline. In Jenkins or GitLab, you can create specific test suites that only run the tests tagged with @a11y. This reduces the runtime of the tests through selection and ensures that accessibility tests are not overlooked.

An important point when running such tests in parallel in CI/CD pipelines is test data management. To ensure reliable results, make sure that the tests are run in a controlled environment with consistent and non-conflicting test data. For example, two tests could conflict if one is running that searches for a specific data set while another test is deleting that very data set.

Installing cypress-grep

Run the following command to install the plugin:

	
	npm install --save-dev cypress-grep
	

Configuration

After installation, you need to enable cypress-grep in your Cypress configuration.

Open or create the file cypress/support/e2e.js and add the following line:

	
	import 'cypress-grep'  // cypress-grep initialisieren
	

To make sure that the tests can be filtered by tags, you can add an environment variable in your cypress.config.js (or.ts for TypeScript):

	
	module.exports = {
		  env: {
		    grepFilterSpecs: true,  // Filtert nur Spezifikationen, die das Keyword enthalten
		  },
		}
	

Tagging tests

In your test code, you can tag tests to categorise them more easily. For example, you can tag your accessibility tests with a special tag:

	
	describe('Accessibility Tests', { tags: '@a11y' }, () => {
		  it('should run accessibility tests on all pages', function () {
		    // Dein Testcode hier
		  })
		  it('should check accessibility for mobile viewports', { tags: '@a11y' }, function () {
		    // Test für mobile Barrierefreiheit
		  })
		})
		describe('E2E Tests', { tags: '@e2e' }, () => {
		  it('should perform full E2E test', function () {
		    // E2E Testcode
		  })
		})
	

In this example, tests with the tag @a11y are marked as accessibility tests and tests with @e2e are marked as end-to-end tests.

Running tests with grep

You can now run only tests with certain tags in the command line. For example, to run only the accessibility tests:

	
	npx cypress run --env grep=@a11y
	

To run E2E tests marked @e2e:

	
	npx cypress run --env grep=@e2e
	

Further information

Testing the initial pages when loading is only a first step. Ideally, you should also test every interaction in which, for example, extra elements are loaded, feedback such as ‘toasties’ appears, localisation or other adjustments.

For iterative integration, you can also use different stages for implementation. Initially, only test for critical impacts or a lower WCAG level. Then increase the test level with each iteration.

Examples

	
	// just critical impacts
		  cy.checkA11y(null, {
		    includedImpacts: ['critical']
		  })
		// filter on A-Level instead of AA or AAA
		cy.checkA11y(null,
		  { runOnly: { type: 'tag', values: [wcag2a], }
		 })
	

Blocking the pipeline in the event of an error could also be prevented for the time being. However, it is important that you also include the whole thing in an impediment backlog / technical debt area when using these practices. In combination with good planning, this can ensure that these points are not forgotten.

Accessibility testing and its limitations: why manual testing is essential

Automated accessibility tests cover many, but by no means all, aspects of accessibility. It is estimated that such tests can detect a maximum of 25% of potential accessibility issues. For example, an automated test can detect whether an image has an alternative text, but not whether this text is meaningful and descriptive. This is where manual tests come into play, allowing for deeper analysis.

A practical example: a website can meet all formal accessibility requirements, such as correct ARIA labels and sufficient contrast. Nevertheless, the user experience (UI/UX) for users with disabilities can be limited if the navigation is too complex or not intuitive. Manual testing by experts or users with disabilities is necessary to identify and fix such problems.

Conclusion

Automated accessibility testing with cypress-axe is an effective way to identify and fix accessibility issues in digital projects at an early stage. By integrating tools such as the Cypress XRay for Jira plugin and cypress grep, these tests can be seamlessly integrated into the development process and executed in a targeted manner. Nevertheless, the importance of manual testing and close collaboration with UI/UX professionals remains essential to ensure comprehensive accessibility. By combining these tools and best practices, you can significantly improve the time it takes to perform an initial accessibility check of your projects. In addition, you are ensuring that your projects are easy to maintain.

You can find more exciting topics from the world of adesso in our previous blog posts.