Accessibility testing with Cypress

Effective user experience (UX) design is a key factor in creating compelling software products. UX considers the quality of interaction that users have with a product and takes the user’s point of view as the most sacred thing in software and product design. A great UX includes accessibility, which ensures that software is inclusive and usable by the widest possible audience.

This article will demonstrate how you can use Cypress and continuous integration (CI) for accessibility testing and also how you can fix accessibility issues in a React frontend application.

Prerequisites

The following are required to successfully complete this tutorial:

1. Node.js installed on your system
2. A CircleCI account
3. A GitHub account and an understanding of Git
4. Basic knowledge of JavaScript and Node.js
5. Knowledge of how CI/CD pipelines work
6. A clone of our demo app

What is accessibility testing?

Accessibility testing ensures that websites and mobile applications are usable by as many people as possible and especially to individuals who have disabilities, such as hearing loss, visual impairment, and cognitive or physical conditions.

The purpose of an accessibility test is to ensure that the application under test is usable by individuals living with disabilities. It is a subset of usability testing, in which people of all abilities and disabilities are considered.

Accessibility testing ensures that apps are compatible with screen readers, speech recognition software, and screen magnifiers. The importance of this testing is to ensure usability and inclusivity.

During accessibility testing, we will often test the following:

• Content (e.g. buttons, labels, anchor tags, and text alignment)
• Global code (e.g. viewport zoom, landmarks, and linear content flow)
• Keyboard
• Images, headings, lists, controls, tables, and forms
• Media (e.g. video and audio)
• Appearance
• Animation
• Color contrast

In the following section, we will examine how to use Cypress for accessibility testing. We will run the login form we cloned previously then use an npm module, axe-core, to check for violations and record them. Later, we’ll fix the violations and re-run the tests on Cypress to ensure the login form meets all accessibility standards.

Tip: Axe is an accessibility testing engine for websites and other HTML-based user interfaces. It’s fast, secure, lightweight, and was built to seamlessly integrate with any existing test environment so you can automate accessibility testing alongside your regular functional testing. In our case axe-core will be used with Cypress as an npm module.

Using Cypress for accessibility testing

Cypress is a modern end-to-end (E2E) testing framework for running front-end E2E tests. It makes automated testing quicker as it executes on the browser, unlike Selenium related front-end testing frameworks. Cypress performs all actions using JavaScript, which gives you more control over your pages and also increases your access to the JavaScript Document Object Model (DOM).

In this tutorial, you will use axe-core, an accessibility engine for automated web UI testing, along with Cypress as the testing framework. Axe works by identifying Web Content Accessibility Guidelines (WCAG) violations and throwing errors by comparing the implementation to the WCAG issues. From the analysis, axe-core then determines whether the issues require a manual review or they pass the criteria. Any violations are recorded as failures.

To determine whether the sample app passes your Cypress accessibility tests, you will test the interface of the login form above against a set of guidelines using Cypress and axe-core. If the interface design violates the accessibility standard, the violations will be recorded, and the test will fail. The tests will pass once you have validated that all accessibility policies are followed.

In the following sections, you’ll see how this works in an actual test.

How to test for accessibility with Cypress

In this section, you’ll go over how to:

• Configure Cypress
• Integrate axe-core to work with Cypress
• Create test cases
• Run tests
• Fix accessibility issues

You will learn three ways to check for accessibility issues, namely:

1. Using Cypress’s interface
2. Through a browser extension
3. Headless mode

Let’s start by installing and configuring Cypress.

Configuring Cypress

Open the project you cloned before and run the following command to install Cypress.

npm install cypress

Globally installing a package has a few drawbacks, which is why I recommend installing it in the working environment.

After installing Cypress, the next thing you should do is launch it with the following command:

npx cypress open

After installing Cypress version 10, you should see the following window. It is important to note that Cypress versions lower than version 10 are different in appearance. You can read the changelog for more information.

Click E2E testing, and Cypress will do the configuration under the hood for you.

Next you will be prompted to modify the configuration file, which you can accept as defaults. You will also need to choose a browser; for our case we will select Chrome.

Now you get the option to create your first spec. Because this is the first time, you will be prompted to either select a scaffold example spec or create a new empty spec. For this tutorial, create a new empty spec and proceed.

After running your new spec file, you should see passing tests in a window similar to the one below.

Congratulations, you have successfully installed and configured Cypress on your Chrome browser for end-to-end testing. In the next section, you will learn how to integrate axe-core and write test cases to check for accessibility issues.

Integrating axe-core to work with Cypress

To integrate Cypress with axe-core, you will first need to run the following command to install axe-core alongside cypress-axe.

npm install cypress-axe axe-core

Next, locate the configuration file created in your Cypress project in the directory location cypress/support/e2e.js and add the following import.

import "cypress-axe";

The file cypress/support/e2e.js is automatically processed and loaded before your test files. Including the import in the support file will ensure that cypress-axe is imported every time our tests run.

Now, let’s update the spec file so that the test case checks for accessibility issues on our registration form.

But first, let’s look at what the autogenerated test case in cypress/e2e/spec-copy-1.cy.js does.

describe("empty spec", () => {
  it("passes", () => {
    cy.visit("https://example.cypress.io");
  });
});

In the above test suite, Cypress provides a command visit() that you use to visit a remote URL.

You can read about the visit command in the Cypress documentation.

Now modify the spec.cy.js file to look like as follows.

/// <reference types="cypress" />

describe("Sign Up Accessibility test suite", () => {
  before(() => {
    cy.visit("http://localhost:3000");
    cy.injectAxe();
  });

  it("Check entire page for a11y issues", () => {
    cy.checkA11y();
  });
});

The first line in the code snippet, which appears to be a comment but is not, makes Cypress auto-suggestions available to you when writing the tests.

Tip: Autosuggestions enable the IDE to automatically suggest Cypress syntax to users as they are writing the test code instead of having to remember every Cypress syntax.

You want Cypress to visit the page you’re testing before executing your test cases, so you provide the URL of our page running on localhost port 3000.

The axe-core runtime is injected into the page under test by the command cy.injectAxe(). According to the documentation, this must be run after a call to cy.visit() and before the checkA11y() command.

Your test suite is complete; let’s see if we can cover any accessibility issues. Start the app and re-open the Cypress interface using the commands listed below.

Running accessibility tests using the Cypress interface

Start the app with the command npm start and head to the Cypress command log.

Click on the spec’s name below to run our accessibility test.

If you notice that our tests fail, some accessibility standards have been violated. Let us now take a closer look at the error messages we received from Axe framework.

According to the issues raised, four accessibility violations were discovered. We have just checked for accessibility issues using the Cypress interface; next, we will use the browser extension option to check for issues.

Running accessibility tests using the axe-core browser extension

To install the axe-core browser extension, go to the Axe website and click Install free Chrome extension.

Once you’ve installed the extension, navigate to Chrome dev tools and select axe dev tools. You should see the following:

In the plugin, you can choose either option. For this tutorial, choose the option to scan all of the page.

You can find more information and tips on addressing an accessibility issue by clicking on any of the issues as shown here:

Now, Let’s finally look at how we can achieve the same checks of accessibility violations using Cypress in a headless way.

Running Cypress accessibility tests in headless mode

In contrast to the first method, in which Cypress provided a browser interface, headless mode simply means that you run a command and get the results printed on a terminal.

To do this, run the following command:

npx cypress run --headless

As with the other two methods, you can expect the tests to fail. In the following section, you will address the violations and ensure that your login page is accessible.

Fixing accessibility issues

You can learn the following from the failed accessibility tests:

Issue 1: Form elements must have labels

Issue 2: Document should have one main landmark

Issue 3: Page should contain a level-one heading

Issue 4: All page content should be contained by landmarks

Issue 5: Elements must have sufficient color contrast

So how do we fix these accessibility violations?

The first issue can be resolved by assigning unique ids to the form input elements. To fix this, you will provide unique IDs for each form input field.

The second issue is a missing major landmark. A landmark serves as a navigation point to the page’s main content. To address the second issue, you need to add the role attribute with the value main to the main div with the ID root in public/index.html. This will fix the fourth issue as well.

To address the third issue, you need to ensure that the form contains an h1 tag.

As you may have noticed, our form’s submit button lacks clear text; to remedy this, you need to make the text visible by changing its color. This fixes the fifth issue.

I have commented on the correct fixes for all of the accessibility issues in the main application file, src/App.js. Proceed with the modifications and then recheck for problems using any methods discussed.

Rerun the checks. This time, all of your tests should pass.

Note: Our application with the accessibility issues can be found on the main branch of the sample repository. The fixes can be found on the fix/accessibility-violations branch. To run the tests that are passing therefore you will need to run the tests in fix/accessibility-violations branch and not the ones on the main branch.

It’s green! You can be confident that the registration form is usable and accessible to users of varying abilities. In the following section, you will learn how to integrate CircleCI and set up a continuous integration (CI) pipeline to automate running accessibility tests on every change to your application.

Configuring CircleCI for accessibility testing

In this section, you will set up a CI pipeline with CircleCI to run automated tests. If you haven’t already signed up for a free CircleCI account, take a moment to do that now.

To get started, create a .circleci directory in the root of your project directory and add a configuration file named config.yml. Once this is done, add the following configuration to set up a workflow execute our tests.

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/run:
          start: npm start
          wait-on: 'http://localhost:3000'
          command: npx cypress run --headless

To simplify the configuration, you’re using the Cypress orb in the initial setup. Orbs are parameterized packages of CircleCI config that allow you to easily integrate popular tools and services into your pipelines with minimal setup.

Note: To use the Cypress CircleCI orb in your project, you will first need to allow using uncertified orbs: Settings > Security > Allow uncertified orbs.

In the cypress/run job, you first start the server (as you do when running tests locally) and then run the cypress command once it’s up and running.

Next, you need to create a repository (we will use GitHub in this example, but you can also use GitLab or Bitbucket if you prefer) and push the project with your changes to the repository. Then, go to CircleCI and follow the steps below to create a pipeline:

Step 1: Select the project you just pushed to GitHub from the CircleCI projects tab, then click Set Up Project.

Step 2: In the pop-up window, enter the name of your project’s branch where the CircleCI configuration lives. In our case it is main. Click Set Up Project.

Step 3: The workflow run will begin shortly, and your checks should now pass with a green success badge, as shown.

And just like that, you have successfully set up a CircleCI pipeline to run your automated accessibility tests. Now, whenever you update your project, CircleCI will automatically validate that the changes meet the accessibility standards you specified. Consider adding additional test types for an ever more robust pipeline.

Conclusion

In this tutorial, you learned about accessibility testing, what it is, and why it is essential for a product’s success. You learned how you can set up accessibility testing with Cypress to test your frontend applications. You learned various ways to find accessibility violations and also how to fix them. Finally you learned how to configure CircleCI to run accessibility tests on every change to your application.

Accessibility testing ensures that digital products are inclusive and usable by all. By incorporating accessibility testing into your CI/CD pipeline, you can proactively identify and address accessibility issues early in the development process, resulting in more user-friendly software that reaches a wider audience. Sign up for your free CircleCI account and start delivering more inclusive software today.

I hope you found this post as enjoyable as I did when preparing it. Until the next one, keep learning!

---

Waweru Mwaura is a software engineer and a life-long learner who specializes in quality engineering. He is an author at Packt and enjoys reading about engineering, finance, and technology. You can read more about him on his web profile.

Read more posts by Waweru Mwaura