Testing can mean many different things depending on who is doing it and wherein a process is being performed. The programmers, administrators, users, and consultants all have something different in mind when testing. A dedicated tester can often feel lost in the competing interpretations. To be effective, however, a tester needs a specific job description. These five goals of software testing are an excellent basis.
Most misunderstood about testing is the primary objective. If you think it is to find defects, then you are wrong. Defects will be found by everybody using the software. Testing is a quality control measure used to verify that a product works as desired. Testing provides a status report of the actual product in comparison to requirements (written and implicit). Simultaneously, this is a pass/fail listing of product features; in detail, it includes confidence numbers and expectations of defect rates throughout the software.
This is important since a tester can hunt bugs forever yet not say whether the product is fit for release. Having a multitude of defect reports is of little use if there is no method to value them. A corporate policy needs to be in place regarding the quality of the product. It must state what conditions are required to release the software. The tester’s job is to determine whether the software fulfills those conditions.
Not everything can be tested. Not even a significant subset of everything can be tested. Therefore, testing needs to assign effort reasonably and prioritize thoroughly. This has been by no means a simple topic. Generally, you’d like to have every feature covered with at least one valid input case. This ensures at least a baseline utility to the software.
Beyond the baseline, you’ll need to test further input permutations, invalid input, and non-functional requirements. In each case, the realistic use of the software should be considered. Highly present and frequent use scenarios should have more coverage than infrequently encountered and specialty scenarios. Overall you target a wide breadth of coverage with depth in high use areas and as time permits.
Exactly what was tested and how it was tested are needed as part of an ongoing development process. In many environments, such proof of activities is required as part of a certification effort or simply as a means to eliminate duplicate testing efforts. This shouldn’t mean extra documentation; it simply means keeping your test plans clear enough to be reread and understood.
You will have to agree on the documentation methods; every team member should not have their own. Not all features should be documented the same way; however: several different methods will likely be employed. Unfortunately, there aren’t many commonly agreed principles in this area, so in a way, your kind of on your own.
Tests must balance the written requirements, real-world technical limitations, and user expectations. Regardless of the development process being employed, there will be a lot of unwritten or implicit requirements. It is the job of the tester to keep all such requirements in mind while testing the software. A tester must also realize they are not a user of the software; they are part of the development team. Their personal opinions are but one of many considerations. Bias in a tester invariably leads to a bias in coverage.
The end user’s viewpoint is obviously vital to the software’s success, but it isn’t all that matters. If the needs of the administrators can’t be met, the software may not be deployable. If the needs of the support team aren’t met, it may be unsupportable. If the needs of marketing can’t be met, it may be unsellable. The programmers can’t be ignored; every defect must be prioritized concerning their time limits and technical constraints.
The discovery of issues should not be random. Coverage criteria should expose all defects of a decided nature and priority. Furthermore, later surfacing defects should be identifiable as to which branch in the coverage it would have occurred, and can thus present a definite cost in detecting such defects in future testing.
This goal should be a natural extension to having traceable tests with priority coverage. It reiterates that the testing team should not be a chaotic black box. Quality control is a well structured, repeatable, and predictable process. Having clean insight into the process allows the business to better gauge costs and to better direct the overall development.