The testing pyramid: Strategic software testing for Agile teams

The testing pyramid model untangles the complexity of software testing by fitting it into an efficient hierarchical structure. By focusing on unit tests at the base, integration tests in the middle, and end-to-end tests at the top, the testing pyramid ensures that most testing efforts are spent on tests that are fast, reliable, and easy to maintain. This allows for quicker iterations, improved code quality, and more stable releases.

This article delves into the intricacies of the testing pyramid, exploring its structure, benefits, and pivotal role in enhancing continuous integration and continuous delivery (CI/CD) processes. You will also learn practical strategies for effectively implementing the testing pyramid into your Agile software development workflows.

What is the testing pyramid?

Although various people have shaped and popularized the concept of the testing pyramid, its most notable proponent was software engineer and Scrum pioneer, Mike Cohn. He introduced the Testing Pyramid in his 2009 book Succeeding with Agile: Software Development Using Scrum.

Cohn proposed the testing pyramid could address challenges in managing automated testing in software development, particularly when following Agile methodologies. The key idea was to encourage developers and testers to think about different levels of testing and to balance their efforts across these levels:

• Unit tests: Forming the base of the pyramid, these tests focus on a single functionality and small units of code. They are typically numerous but quick to write and execute.

  Unit tests should be run frequently, ideally as part of an automated continuous integration process, to ensure that new changes do not break existing functionality and to maintain a high standard of code quality throughout the development lifecycle.

• Integration tests: Comprising the middle of the pyramid, these tests check the interaction between different modules or external systems. Compared to unit tests, they are fewer in number, more expensive, and slower to execute.

  Integration tests should be run less frequently than unit tests, typically at key points during the development cycle such as after significant features or changes have been integrated into the main branch of the project. This ensures that while the more detailed and granular unit tests are run with every small change, integration tests verify the system’s overall coherence and functionality at critical stages, balancing thoroughness with the need for efficiency in the development process.

• End-to-end (E2E) tests: Sitting atop the pyramid, E2E tests involve checking the entire application as end users would experience it, including user interfaces, APIs, databases, and other services. They are the most complex and have the longest execution time.

  E2E tests should be run at significant milestones, such as before releases. Given their comprehensive nature and broader scope, E2E tests are typically more time-consuming and resource-intensive than both unit and integration tests. Therefore, they are run less frequently to validate the entire application’s workflow from start to finish under conditions that mimic real-world usage.

The following diagram shows the pyramid’s three key layers and their focal areas in the testing process.

As open-box tests, the constrained and focused unit tests provide the tester with complete awareness of the code’s internal workings and structure. Comparatively, developers often conduct integration and E2E tests without detailed knowledge of each component’s internal workings — in other words, as closed-box tests.

The testing pyramid provides a roadmap for quality assurance throughout the Agile software development lifecycle.

Benefits of the testing pyramid

The testing pyramid encourages a balanced and targeted approach to automated testing. It aims to prevent over-reliance on any one type of test, such as E2E tests, which are often more brittle and expensive to maintain. The model helps teams catch more bugs at the lower levels of the pyramid, where they are cheaper and easier to fix. Simultaneously, it ensures sufficient E2E testing to confirm that the system functions correctly.

Furthermore, this model promotes efficient resource allocation. You perform numerous low-cost unit tests but fewer complex and pricey E2E tests. The pyramid structure facilitates faster feedback — starting with numerous unit tests lets you quickly identify and resolve issues.

Finally, following the testing pyramid model ensures comprehensive test coverage, from individual components to the system as a whole. This also aligns closely with Agile principles, enhancing software development quality while maintaining speed and adaptability.

The testing pyramid in practice

To better understand how to apply the testing pyramid model, consider the following practical example: developing a project management tool.

Unit testing example

The unit testing stage of our project management tool involves testing individual components or functions in isolation. For example:

• Task creation logic: Validating the feature for adding a new task to ensure it incorporates all required details, such as title, description, and deadline
• User authentication methods: Ensuring the user login feature ensuring correctly validates user credentials and handles errors like incorrect passwords or usernames
• Notification system: Checking that the notification system generates the correct alerts for task deadlines or updates

These tests would mock or stub dependencies like database calls or external services to focus solely on the logic of the component being tested. They are automated and run frequently during development to catch and fix bugs early.

Integration testing example

At the integration testing stage, the focus shifts to how different components of the project management tool interact. For example:

• Task and calendar integration: Test how new tasks appear in the project calendar, verifying that tasks show up on the correct dates and that updates such as changing a due date are reflected in the calendar.
• User permissions: Ensure that changes made by one user (like updating a task status) are correctly reflected for other users with access to the same project, testing the integration between user roles, task management, and real-time updates.

These tests involve a combination of components and require a more comprehensive setup, possibly including real database access or API calls. They are run less frequently than unit tests, typically after a group of components has been developed or updated.

E2E testing example

E2E testing evaluates the project management tool’s entire workflow. In the context of a project management tool, this could involve:

• Project lifecycle: Test the application’s ability to handle a project from inception to conclusion by creating a new project, adding team members, creating tasks, updating task status, and archiving or completing the project.
• Collaboration features: Check the tool’s collaborative capabilities and real-time update features by simulating multiple users working on the same project, including assigning tasks, commenting on tasks, and tracking project progress.

E2E tests typically use tools that simulate user interactions with the application’s interface, such as clicking buttons, filling out forms, and navigating through the application. They require a fully deployed application environment and are run at critical milestones, such as before a release or after significant features have been added to the application.

CI/CD and the testing pyramid

CI/CD automates the process of building, testing, and deploying your code, giving you complete control over how and when your tests and other development tasks are executed. The iterative nature of CI/CD processes means they integrate perfectly with the testing pyramid model, particularly in Agile environments.

A typical CI/CD pipeline executes unit tests on every commit to a development branch, providing immediate feedback to help developers catch issues early in the development cycle.

Integration tests typically run after unit tests have successfully passed and before a merge into the main branch, ensuring that different components work well together before significant changes are integrated into the broader application.

End-to-end (E2E) tests are usually executed after all changes have been merged into the main branch and before deployment to staging or production environments, serving as a final verification that the application meets all requirements and functions correctly in an environment that closely mimics the production setting.

This approach is a boon to Agile teams, facilitating rapid development and deployment. The pyramid’s testing strategy ensures each deployment’s speed and reliability — crucial for Agile teams aiming to consistently deliver high-quality software.

For more information on how to orchestrate tests across your development process, read The path to production: How and where to segregate test environments.

Implementing the testing pyramid in your development workflows

Integrating the testing pyramid into software development workflows requires a strategic approach. Here are some tips for implementing this model in your projects:

• Start with unit testing: A robust suite of unit tests forms the pyramid’s foundation. Encourage developers to write unit tests alongside new code, adhering to practices like test-driven development (TDD).

• Focus on integration tests: After you have established unit testing and TDD as a fundamental practice, develop integration tests to ensure that various components of the application work well together. These tests bridge the gap between unit testing and E2E testing.

• Strategically implement E2E tests: Due to their complexity and resource demands, E2E tests require careful planning and implementation. You should only implement them to cover the application’s critical user journeys and major functionalities.

• Regularly review and adjust: Continuously assess the test suite’s effectiveness and efficiency. Prepare to adjust the balance of tests as the project evolves, adding more tests where needed and refining existing ones.

• Take advantage of automation: Use automation tools to streamline testing processes, especially in CI/CD pipelines. Tools like CircleCI can automate test execution, running them consistently and efficiently.

• Educate and collaborate: Make sure all team members understand the importance and role of each tier in the pyramid. Foster a collaborative environment where developers, Quality Assurance (QA) engineers, and other stakeholders work together to maintain and improve the test suite.

Conclusion

The testing pyramid is a crucial model in Agile software development, providing a simple, clear, and efficient approach to testing. Its tiered structure of unit, integration, and E2E tests ensures a balanced, comprehensive testing strategy that aligns closely with Agile principles.

The model maximizes resource efficiency and promotes rapid development by emphasizing a high volume of low-cost unit tests and progressing to fewer, more complex tests. Furthermore, it offers critical support for CI/CD workflows, helping maintain the agility and speed that modern software development demands.

To get started using the testing pyramid to optimize your testing process and improve your software quality, sign up for a free CircleCI account today.