The Fundamentals of Software Testing
“The Fundamentals of Software Testing” is the first module of the ISEB Foundation course and examination syllabus. ISEB recommend that Foundation students spend 20{3bb2a8e703be8d5bb7fc1289a915cd39229c5bcd006c8cdf059732c7e19a8eab} of their study time on the Fundamentals course section. This is also the title of the opening chapter in the official ISEB Software Testing manual.
Understanding the fundamentals of software testing (ST) is clearly crucial to achieving proficiency according to the ISEB professional standards. But what, precisely, are these “fundamentals”, and what do you need to know in order to pass the ISEB Foundation examination?
Why is it necessary?
According to Peter Morgan (freelance testing practitioner and member of the ISEB Accreditation and Examination panel), an unnecessarily high proportion of IT systems fail to fulfil expectations – or fail to work at all – because of insufficient ST prior to release.
ST cannot guarantee against software problems or even failure, but it can minimise the risk of faults developing once the software is in use. In order to understand why ST is necessary, it is important to consider the causes and impact of IT systems failure, and the crucial role that testing plays in Quality Assurance.
The ISEB Foundation training-courses focus on five learning requirements:
o The potential impact of an IT systems failure;
o Causes/effects of software failure;
o Why ST is necessary;
o ST versus Quality Assurance;
o Basic industry terminology.
What is ST?
ST is a method of identifying faults in the product before it is released; this includes defects in the software code and its execution, as well as any potential gap between what the code is supposed to do and what it actually does.
The ISEB ST Foundation exam requires students to:
o Know the basic objectives of ST;
o Understand how these objectives apply in the context of creating, using and supporting software (including the difference between testing and ‘debugging’, and the different uses of static and dynamic testing).
General principles
The ISEB courses introduce students to 7 software testing principles:
o Testing can show that problems exist, but not that problems do not exist.
o Testing covers a limited range of sample situations, and is not comprehensive.
o The earlier a problem is found, the less it costs to fix.
o Defects will tend to cluster around specific areas (e.g. system complexity, or staff inexperience). These areas can be particularly targeted by testers.
o A set of tests, run repeatedly, will have diminishing effect.
o The type, design and focus of testing will vary according to the software tested.
o A test that does not reveal errors does not prove that the software is error-free.
Basic process
Software testing is not simply a case of reviewing documentation or running a software program to check for bugs. To ensure that the appropriate tests are operated effectively and return value on the invested resources, it is crucial that software testers carry out five basic activities, which the ISEB ST guidance links to form a ST process:
1. Test planning and control
a. Test planning defines what will be tested, how it will be tested, and who we will know when the testing is complete.
b. Test control is an iterative activity that compares the progress against the plan, and adjusts the plan as required.
2. Test analysis and design
a. Test analysis considers the specific conditions to be tested, and how these test conditions are to be combined into test cases.
b. Test design describes the expected outcome of the test, so that testers can easily recognise when a fault has occurred.
3. Test implementation and execution
Execution represents the most visible testing activity, and encompasses setting up the tests, generating test data, logging test activities and analyzing the test results.
4. Evaluating exit criteria and reporting
The test “exit criteria” (established during “Test planning and control”) indicate that testing is complete.
5. Test closure activities
o Ensure that all documentation has been correctly completed
o Close down and archive the test environment, infrastructure and testware
o Hand over testware to maintenance team
o Log lessons learned
The psychology of testing
Selecting the appropriate individual(s) to perform the tests, and communicating the results of the tests to the developers, requires a basic understanding of the psychology of testers and developers.
The ISEB guidance provides a hierarchy of ‘independence’ – that is, a list of individuals who could potentially test the software, gradated according to their perceived level of detachment from the development process:
o The authors of the code
o Members of the same development team as the authors of the code
o A different group within the same organization
o A different organization
There are varying advantages and disadvantages to individuals at different levels in the hierarchy performing the software tests. For example, the developer might be able to resolve a difficulty more quickly, but might be also be less likely to spot the difficulties in the first place.
In addition to the independence/detachment of the testers from the software development process, it is important to consider how testers can communicate the results of the test to the developers. The ISEB Software Testing Foundation exam requires students to explain strategies for providing courteous feedback on defects, for example:
o Focus on delivering quality, rather than pointing out mistakes;
o Test reports are not directed towards people but towards products;
o Discuss problems openly, and attempt mutual understanding;
o At the end of a discussion, confirm that both parties understand what has been agreed.
Summary
Understanding the fundamental is crucial to passing the ISEB Software Testing Foundation examination. During an accredited ISEB Foundation course, approximately two and a half hours should be devoted to studying what testing is, why it is necessary, and the fundamentals principles and processes behind a successful test performance.
