The different types of testing in software | Atlassian (2024)

It's important to make the distinction between manual and automated tests. Manual testing is done in person, by clicking through the application or interacting with the software and APIs with the appropriate tooling. This is very expensive since it requires someone to setup an environment and execute the tests themselves, and it can be prone to human error as the tester might make typos or omit steps in the test script.

Automated tests, on the other hand, are performed by a machine that executes a test script that was written in advance. These tests can vary in complexity, from checking a single method in a class to making sure that performing a sequence of complex actions in the UI leads to the same results. It's much more robust and reliable than manual tests –but the quality of your automated tests depends on how well your test scripts have been written.If you're just getting started with testing, you can read our continuous integration tutorial to help you with your first test suite. Looking for more testing tools? Check out theseDevOps testing tutorials.

Automated testing is a key component ofcontinuous integrationandcontinuous deliveryand it's a great way to scale your QA process as you add new features to your application. But there's still value in doing some manual testing with what is called exploratory testing as we will see in this guide.

1. Unit tests

Unit tests are very low level and close to the source of an application. They consist in testing individual methods and functions of the classes, components, or modules used by your software. Unit tests are generally quite cheap to automate and can run very quickly by a continuous integration server.

2. Integration tests

Integration tests verify that different modules or services used by your application work well together. For example, it can be testing the interaction with the databaseor making sure that microservices work together as expected. These types of tests are more expensive to run as they require multiple parts of the application to be up and running.

3. Functional tests

Functional tests focus on the business requirements of an application. They only verify the output of an action and do not check the intermediate states of the system when performing that action.

There is sometimes a confusion between integration tests and functional tests as they both require multiple components to interact with each other. The difference is that an integration test may simply verify that you can query the database while a functional test would expect to get a specific value from the database as defined by the product requirements.

4. End-to-end tests

End-to-end testing replicates a user behavior with the software in a complete application environment. It verifies that various user flows work as expected and can be as simple as loading a web page or logging in or much more complex scenarios verifying email notifications, online payments, etc...

End-to-end tests are very useful, but they're expensive to perform and can be hard to maintain when they're automated. It is recommended to have a few key end-to-end tests and rely more on lower level types of testing (unit and integration tests) to be able to quickly identify breaking changes.

5. Acceptance testing

Acceptance tests are formal tests that verify if a system satisfies business requirements. Theyrequire the entire application to be running while testing and focus on replicating user behaviors. But they can also go further and measure the performance of the system and reject changes if certain goals are not met.

6. Performance testing

Performance tests evaluate how a system performs under a particular workload.These tests help to measure thereliability, speed, scalability, and responsiveness of an application. For instance, a performance test can observe response times when executing a high number of requests, or determine how a system behaves with a significant amount of data. It can determine if an application meets performance requirements,locate bottlenecks, measure stability during peak traffic, and more.

7. Smoke testing

Smoke tests are basic tests that check the basic functionality of an application. They are meant to be quick to execute, and their goal is to give you the assurancethat the major features of your system are working as expected.

Smoke tests can be useful right after a new build is made to decide whether or not you can run more expensive tests, or right after a deployment to make sure that they application is running properly in the newly deployed environment.

To automate your tests, you will first need to write them programmatically using a testing framework that suits your application.PHPUnit,Mocha,RSpecare examples of testing frameworks that you can use for PHP, Javascript, and Ruby respectively. There aremany optionsout there for each language so you might have to do some research and ask developer communities to find out what would be the best framework for you.

When your tests can be executed via script from your terminal, you can have them be automatically executed by a continuous integration server like Bamboo or use a cloud service like Bitbucket Pipelines. These tools will monitor your repositories and execute your test suite whenever new changes are pushed to the main repository.

The more features and improvements go into your code, the more you'll need to test to make sure that all your system works properly. And then for each bugyou fix, it would be wise to check that they don't get back in newer releases. Automation is key to make this possible and writing tests sooner or later will become part of your development workflow.

So the question is whether it is still worth doing manual testing? The short answer is yes and it might be best to perform exploratory testing to uncover non-obvious errors.

An exploratory testing session should not exceed two hours and should have a clear scope to help testers focus on a specific area of the software. Once all testers have been briefed, various actions should be used to check how the system behaves.

To finish this guide, it's important to talk about the goal of testing. While it's important to test that users can actually use an application (they can log in and save an object), it is equally important to test that an application doesn't break when bad data or unexpected actions are performed. You need to anticipate what would happen when a user makes a typo, tries to save an incomplete form, or uses the wrong API. You need to check if someone can easily compromise data or gain access to a resource they're not supposed to. A good testing suite should try to break your app and help understand its limit.

And finally, tests are code too! So don't forget them during code review as they might be the final gate to production.

Atlassian’s Open DevOps provides an open toolchain platform that allows you to build a CD-based development pipeline with the tools you love. Learn how Atlassian and third-party tools can integrate testing in your workflow with our DevOps testing tutorials.