Test automation is the practice of automatically reviewing and validating a software product, such as a web application, to make sure it meets predefined quality standards for code style, functionality (business logic), and user experience.
Testing practices typically involve the following stages:
- Unit testing: validates individual units of code, such as a function, so it works as expected
- Integration testing: ensures several pieces of code can work together without unintended consequences
- End-to-end testing: validates that the application meets the user’s expectations
- Exploratory testing: takes an unstructured approach to reviewing numerous areas of an application from the user perspective, to uncover functional or visual issues
The different types of testing are often visualized as a pyramid. As you climb up the pyramid, the number of tests in each type decreases, and the cost of creating and running tests increases.
Historically, all testing within the pyramid was performed manually. This was a slow, expensive, and error-prone process until automated testing tools were created.
Today, nearly all unit tests are fully automated and unit testing automation is considered a best practice. Integration tests are also largely automated and, if not, are typically skipped in favor of more manual end-to-end testing. The current wave of test automation efforts is largely focused on automating the end-to-end layer of the testing pyramid, which reduces the need for integration tests.
Even though automation tools have existed for over a decade, many require coding skills and often result in flaky, brittle tests that are extremely costly to troubleshoot and maintain at scale. Many teams end up creating their own custom test automation frameworks, which makes it difficult and time-consuming to onboard new team members due to the steep learning curve. Custom frameworks also end up requiring their own maintenance and improvements to keep up with the changing technology stack. As a result, most end-to-end testing was a manual process — until now.
As organizations mature their DevOps practices, the need for test automation across the lifecycle is important to unlock the key benefits of DevOps — the ability to build, test, and ship faster and more reliably, streamline incident responses, and improve collaboration and communication across teams. It is no longer an option to have a release build sit with the QA team for several days before developers receive feedback and fix identified issues. QA teams need to align their efforts in the DevOps cycle by ensuring test cases are automated and achieve near 100% code coverage. Environments need to be standardized and the deployment on their QA boxes should be automated. Pre-testing tasks, cleanups, post-testing tasks, etc. should be automated and aligned with the continuous integration cycle.
There are now low-code tools like mabl that make it possible to incorporate reliable and automated end-to-end tests at every stage of the CI/CD pipeline, which helps catch issues much earlier in the development lifecycle. It’s well known that the earlier you detect problems with a release, the faster and cheaper it is to fix them.