The fundamental test process according to ISTQB

Awareness of testing and the need for it has grown steadily among most developers over the years. This means that the question no longer arises as to whether testing is necessary on a larger scale, but the real question is now: "What is the best way to go about testing?"

There are certainly many approaches and theories that have more or less similarities. In this blog entry I would like to specifically describe the testing process according to ISTQB and its phases.

The ISTQB (International Software Testing Qualifications Board) was founded in 2002 and aims to provide standardized training for software testers. Specifically, according to the current status, 12.000 certifications are carried out every quarter. There are already more than 100 certified testers in over 350.000 countries.

There are national agencies around the world, such as the German, Austrian and Swiss Testing Boards, which carry out the certifications.

The process is initiated by the Test planning and control. These two points affect all subsequent phases. Above all, this means that the later phases are subject to the results developed in this phase and that this is also monitored.

The Test Planning includes the definition of the test strategy and the scope to which testing is to be carried out. In order to correctly determine the scope, it is necessary to consider which software components are to be tested and what the specific test goals are. It is advisable to draw up a schedule here as well and to determine the resources required in the course of this in order to be able to plan for them. Test planning also includes setting up test metrics and defining entry and exit criteria.

The test control deals with the tracking of the tests. The test results are measured and analyzed taking into account the parameters from the test planning. Test progress, test coverage and exit criteria are also monitored and documented. If deviations from the plan are identified, it may also be necessary to adjust the plan. In practice, test control should therefore give feedback to test planning so that the scenarios that led to the deviation that occurred can be taken into account in the future.

In the second phase – test analysis and test design – it gets a little more specific. In it, the test basis is examined; the goal is to create a test specification. In order to get valuable information for the test, interviews are conducted with the developers, users, etc. Furthermore, the testability of the test basis and the test objects is evaluated and the test conditions are identified. Based on this information, the design of abstract test cases can begin. Even a prioritization of these test cases can already be carried out at this point. The topic of traceability should also not be forgotten, i.e. the reference from test basis to test cases in both directions. However, not only the test cases are specified in this phase, the test environments are also described in more detail.

In the third phase "realization and implementation“ concrete test cases are derived from the test specification, which contains abstract test cases. If required, these can be combined in test suites. The prioritization from the previous phase is largely applied to the test cases and a test execution plan is created. In order for the tests to be carried out afterwards, the specific test data must be created and made available.

The person responsible in this phase should also pay attention to the test environments and their infrastructure so that there are no delays in the start of the test.

When all test cases have been created, the test execution can start. Regardless of whether the test is carried out manually or automatically, the target and actual results are compared in both variants. If deviations occur, the causes must be analyzed. Regardless of whether there are deviations or not, each test case is logged in detail so that the test can be reproduced later. Certain data are particularly important for logging the test and must not be missing:

• Date, test person, input data used and target data;
• ID/version of test object, test tool and test equipment;
• Possible causes of error, comments on special features.

This data is summarized in the test protocol. If deviations (errors) occur, error reports are created. These reports are sent to the responsible persons, who are in the development department, for example, to be corrected. If the errors have been corrected, a post-test is required, which can also be supplemented by further regression tests to be on the safe side. You are also free to run other (already processed) test cases again.

After the test has been carried out, the Evaluation of the exit criteria and the report. The main purpose is to assess whether the initial criteria have been met and whether further tests should be carried out. The result of this phase is the final test report, which is distributed to the appropriate decision makers.

In the final phase, the test completion, mainly final work is carried out. It is checked whether all work results have been created and delivered and whether open errors and deviations may have to be transferred to change requests. A very important point that often receives little attention is the documentation and archiving of test equipment and the test environment. These could be important for follow-up projects and maintenance, but it also makes it easier to reproduce bugs later. Finally, the knowledge gained can also be discussed in a lessons-learned meeting and improvements for follow-up projects can be developed.

Conclusion

The five phases cover all important testing tasks and give clear instructions as to which order makes the most sense when testing, which documents must be created in the phases and which responsibilities there are in the individual roles. In my opinion, the ISTQB testing process covers all relevant topics and is suitable for medium to large projects. Whether the rather extensive framework of test planning is necessary in small projects should be decided on a case-by-case basis. Depending on the project, the very comprehensive planning could be sensibly reduced in a few points. The test completion can also be somewhat reduced for small projects. If, for example, only small things were changed in a project, the question arises as to whether all environments really have to be archived. Overall, the ISTQB testing process is definitely worth looking at and considering using. However, it should be critically examined whether all areas must be complied with in detail. The test process is only a guide and not the only correct solution, which must be applied one-to-one for each project in order to test successfully.