Accessibility testing with Cypress

Effective user experience (UX) design is a key factor in creating compelling software products. UX considers the quality of interaction that users have with a product and takes the user’s point of view as the most sacred thing in software and product design. A great UX requires accessibility, which ensures that software is inclusive and usable by the widest possible audience.

This article will demonstrate how you can use Cypress and continuous integration (CI) to set up an automated accessibility testing pipeline and also how you can fix accessibility issues in a React frontend application.

Prerequisites

The following are required to successfully complete this tutorial:

1. Node.js installed on your system
2. A CircleCI account
3. A GitHub or GitLab account and an understanding of Git
4. Basic knowledge of JavaScript and Node.js
5. Knowledge of how CI/CD pipelines work
6. A clone of our demo app

What is accessibility testing?

Accessibility testing ensures that websites and mobile applications are usable by as many people as possible, including those who have disabilities such as hearing loss, visual impairment, and cognitive or physical conditions. It is a subset of usability testing, which measures how well users are able to accomplish their goals with a piece of software.

Accessibility testing ensures usability and inclusivity by ensuring that apps are compatible with screen readers, speech recognition software, and screen magnifiers. During accessibility testing, we will often test the following:

• Content (e.g. buttons, labels, anchor tags, and text alignment)
• Global code (e.g. viewport zoom, landmarks, and linear content flow)
• Keyboard inputs
• Images, headings, lists, controls, tables, and forms
• Media (e.g. video and audio)
• Appearance
• Animation
• Color contrast

In the following sections, we will examine how to use Cypress, a popular end-to-end testing framework, for accessibility testing. We will run the login form from a demo application then use an npm module, axe-core, to check for violations and record them. Later, we’ll fix the violations and re-run the tests on Cypress to ensure the login form meets all accessibility standards.

Using Cypress for accessibility testing

Cypress is a modern end-to-end (E2E) testing framework for running front-end E2E tests. It makes automated testing quicker as it executes on the browser, unlike Selenium related front-end testing frameworks. Cypress performs all actions using JavaScript, which gives you more control over your pages and also increases your access to the JavaScript Document Object Model (DOM).

In this tutorial, you will also use axe-core, an accessibility engine for automated web UI testing, alongside Cypress as the testing framework. Axe works by identifying Web Content Accessibility Guidelines (WCAG) violations and throwing errors by comparing the implementation to the WCAG issues. From the analysis, axe-core then determines whether the issues require a manual review or they pass the criteria. Any violations are recorded as failures.

To determine whether the sample app passes your Cypress accessibility tests, you will test the interface of the login form against a set of guidelines using Cypress and axe-core. If the interface design violates the accessibility standard, the violations will be recorded, and the test will fail. The tests will pass once you have validated that all accessibility policies are followed.

How to test for accessibility with Cypress

In this section, you’ll learn how to:

• Configure Cypress
• Integrate axe-core to work with Cypress
• Create test cases
• Run tests
• Fix accessibility issues

You will learn three ways to check for accessibility issues, namely:

1. Using Cypress’s interface
2. Through a browser extension
3. In headless mode

Let’s start by installing and configuring Cypress.

Configuring Cypress

Open the demo project you cloned before and run the following command to install the dependencies.

npm install

After installing dependencies, the next thing you should do is launch Cypress with the following command:

npx cypress open

Click E2E testing, and Cypress will do the configuration under the hood for you. Next, select Start E2E Testing in Chrome.

If this were a project that didn’t already have specs, you would be prompted to modify the configuration file, which you could accept as defaults. Because we’ve already written the first test for the demo app, you won’t be presented with this option.

After running your new spec file, you should see passing tests in a window similar to the one below.

Congratulations, you have successfully installed and configured Cypress on your Chrome browser for end-to-end testing. In the next section, you will learn how to integrate axe-core and write test cases to check for accessibility issues.

Integrating axe-core to work with Cypress

To integrate Cypress with axe-core for a new project, you would first need to run the following command to install axe-core alongside cypress-axe.

npm install cypress-axe axe-core

You don’t need to do this for our example project, since it already exists in package.json and was installed with the rest of the dependencies.

If you’re in a fresh project, locate the configuration file created in your Cypress project in the directory location cypress/support/e2e.js and add the following import.

import "cypress-axe";

The file cypress/support/e2e.js is automatically processed and loaded before your test files. Including the import in the support file will ensure that cypress-axe is imported every time our tests run.

Again, this step has already been completed for the demo project.

Next, let’s update the spec file so that the test case checks for accessibility issues on our registration form.

The demo project has the first spec, spec.cy.js written as:

/// <reference types="cypress" />

describe("Sign Up Accessibility test suite", () => {
  before(() => {
    cy.visit("http://localhost:3000");
    cy.injectAxe();
  });

  it("Check entire page for a11y issues", () => {
    cy.checkA11y();
  });
});

The first line in the code snippet, which appears to be a comment but is not, makes Cypress auto-suggestions available to you when writing the tests.

Note: Autosuggestions enable the IDE to automatically suggest Cypress syntax to users as they are writing the test code instead of having to remember every Cypress syntax.

You want Cypress to visit the page you’re testing before executing your test cases, so you provide the URL of our page running on localhost port 3000. You can read about the visit command in the Cypress documentation.

The axe-core runtime is injected into the page under test by the command cy.injectAxe(). According to the documentation, this must be run after a call to cy.visit() and before the checkA11y() command.

Your test suite is complete. Now let’s see if we can uncover any accessibility issues. Start the app and re-open the Cypress interface using the commands in the next section.

Running accessibility tests using the Cypress interface

In a separate terminal window, start the app with the command npm start and head to the Cypress command log.

Click on spec.cy.js to run your accessibility tests.

If your tests fail, then some accessibility standards have been violated. Let’s take a closer look at the error messages we received from Axe framework.

According to the issues raised, four accessibility violations were discovered. You can use this information to improve your UX and make your application more accessible to users.

We have just checked for accessibility issues using the Cypress interface. Next, we will use the browser extension option to check for issues.

Running accessibility tests using the axe-core browser extension

To install the axe-core browser extension, go to the Axe website and click Install free Chrome extension.

Once you’ve installed the extension, navigate to Chrome dev tools with the project open and select axe dev tools. You should see the following:

In the plugin, you can choose either option. For this tutorial, choose the option to scan all of the page.

You can find more information and tips on addressing an accessibility issue by clicking on any of the issues as shown here:

Now, Let’s finally look at how we run the same tests for accessibility violations using Cypress in a headless way.

Running Cypress accessibility tests in headless mode

In contrast to the first method, in which Cypress provided a browser interface, headless mode simply means that Cypress runs the tests without launching a visible browser window. Instead, you run a command and get the results printed on a terminal.

To do this, run the following command with the application running in another window:

npx cypress run --headless

As with the other two methods, you can expect the tests to fail. In the following section, you will address the violations and ensure that your login page is accessible.

Fixing accessibility issues

You can learn the following from the failed accessibility tests:

Issue 1: Form elements must have labels

Issue 2: Document should have one main landmark

Issue 3: Page should contain a level-one heading

Issue 4: All page content should be contained by landmarks

Issue 5: Elements must have sufficient color contrast

So how do we fix these accessibility violations?

The first issue can be resolved by assigning unique ids to the form input elements. To fix this, you will provide unique IDs for each form input field.

The second issue is a missing major landmark. A landmark serves as a navigation point to the page’s main content. To address the second issue, you need to add the role attribute with the value main to the main div with the ID root in public/index.html. This will fix the fourth issue as well.

To address the third issue, you need to ensure that the form contains an h1 tag.

As you may have noticed, our form’s submit button lacks clear text; to remedy this, you need to make the text visible by changing its color. This fixes the fifth issue.

I have commented on the correct fixes for all of the accessibility issues in the main application file, src/App.js. Proceed with the modifications and then recheck for problems using any methods discussed.

Rerun the checks. This time, all of your tests should pass.

Note: Our application with the accessibility issues can be found on the main branch of the sample repository. The fixes can be found on the fix/accessibility-violations branch. To run the tests that are passing, you will need to run the tests in fix/accessibility-violations branch and not the ones on the main branch.

It’s green! These accessibility tests help build confidence that the registration form is usable and accessible to users of varying abilities. In the following section, you will learn how to integrate CircleCI and set up a continuous integration (CI) pipeline to automate running accessibility tests on every change to your application.

Automating accessibility testing with CI

So far, you’ve learned how to run accessibility tests manually, which is a great starting point for catching issues early in development. However, manual testing alone can be inefficient and inconsistent, especially as your application grows and changes over time. To ensure continuous accessibility compliance, it’s best to integrate these tests into your development pipeline.

In this section, you will set up continuous integration (CI) with CircleCI to run these tests automatically after every change to your application, giving you a continuous and reliable way to ensure that accessibility standards are met throughout the development process. If you haven’t already signed up for a free CircleCI account, take a moment to do that now.

To get started with a new project, create a .circleci directory in the root of your project directory and add a configuration file named config.yml. Once this is done, add the following configuration to set up a workflow execute our tests. You’ll notice this configuration already exists in the demo application.

version: 2.1
orbs:
  cypress: cypress-io/cypress@3.1.2
workflows:
  build:
    jobs:
      - cypress/run:
          start-command: npm start
          cypress-command: npx cypress run --headless

To simplify the configuration, you’re using the Cypress orb in the initial setup. Orbs are parameterized packages of CircleCI config that allow you to easily integrate popular tools and services into your pipelines with minimal setup.

Note: To use the Cypress CircleCI orb in your project, you will first need to allow using uncertified orbs: Settings > Security > Allow uncertified orbs.

In the cypress/run job, you first start the server (as you do when running tests locally) and then run the cypress command once it’s up and running.

Next, you need to create a repository (we will use GitHub in this example, but you can also use GitLab or Bitbucket if you prefer) and push the project with your changes to the repository. Then, go to CircleCI and follow the steps below to create a pipeline.

Step 1: In the organization dashboard, select Create Project and then select your project from the dropdown. CircleCI will automatically detect your configuration, so you can proceed by clicking Create Project

Step 2: Select that you prefer to use an existing config. CircleCI won’t run your pipeline automatically. You’ll need to first push a small, meaningless commit to trigger a pipeline run.

Step 3: The workflow run will begin shortly, and your checks should now pass with a green success badge, as shown.

And just like that, you have successfully set up a CircleCI pipeline to run your automated accessibility tests. Now, whenever you update your project, CircleCI will automatically validate that the changes meet the accessibility standards you specified. Consider adding additional types of tests for an even more robust pipeline.

Conclusion

In this tutorial, you learned about accessibility testing, what it is, and why it is essential for a product’s success. You learned how you can set up accessibility testing with Cypress to test your frontend applications. You learned various ways to find accessibility violations and also how to fix them. Finally you learned how to configure CircleCI to run accessibility tests on every change to your application.

Accessibility testing ensures that digital products are inclusive and usable by all. By incorporating accessibility testing into your CI/CD pipeline, you can proactively identify and address accessibility issues early in the development process, resulting in more user-friendly software that reaches a wider audience. Sign up for your free CircleCI account and start delivering more inclusive software today.

I hope you found completing this tutorial as enjoyable as I did writing it. Until the next one, keep learning!