Why software testing is essential
Could you imagine losing $400 million in 37 seconds? That happened to the ESA in 1996 when their carrier rocket Ariane 5 exploded shortly after take-off. The disaster was caused by a foolish software bug and is one of the biggest software failures in history. The starting point of the disaster was the cast of a double variable of the ground acceleration to another data type. The converted value was too big to be displayed within the given range, which caused an exception in the code. This exception enabled a series of reactions which ended in the self-destruction of Ariane 5 (Auer, 2021; Wunderlich-Pfeiffer, 2015).
The disaster of the Ariane 5 took place more than 25 years ago, and it could be assumed that processes, standards and quality assurance in companies have been improved in the meantime. However, that is not valid for every single company on the globe. In 2015 Starbucks had to close almost all its stores in the USA and Canada for half a day due to a problem with the cash registers. The stores couldn’t proceed with any orders and take payments after an internal failure during a daily system update (Starbucks, 2015).
These two real-world examples show how a lack of quality assurance can lead to financial loss, reputational damage and even ending up fatal. But how can companies prevent such disasters? The key is an appropriate and extensive quality assurance process and mindset within the organisation.
Responsibilities for software quality in projects
Software projects are complex, fast moving and often time critical. This can lead to some trade-offs and shifts of priorities within the project team. The project needs to be managed by a project manager, the requirements need to be defined by requirements engineers, and the corresponding software needs to be developed by the development team. But who is responsible for testing and ensuring that the software meets all its defined criteria? In general, the whole project team is responsible for the overall outcome of the final product, including quality aspects. Depending on the methodology and team setup, the primary responsibility for defining and fulfilling quality aspects can be assigned to roles and people (ISTQB, 2021).
So, how can appropriate quality assurance be set up in projects? This will be answered after presenting some misbeliefs about testing and why this mindset is dangerous for conducting a successful project.
Common misbeliefs about testing
As mentioned above, the importance and responsibility of quality aspects are often not anchored in projects. Although testing and fixing bugs in the early stages is cheaper than in the later stages, as shown in Figure 1, some common misbeliefs and flawed approaches still exist such as the following:
- Dedicated test engineers will cost money and cause delays. They do not add functional value to the software project.
- The developers will test their functionality to ensure that the final product is working as expected.
- Extensive testing will happen at the end when the final product has been developed.
- Testing can be done casually by anyone without any strategy.
- Writing unit tests is enough to release the product; the end users will do end-to-end testing.
The listed examples are from actual projects in different areas and sizes. Every single project is unique, but many share the same problem – insufficient quality level due to a missing professional quality assurance process (Ronald, 2020).
Testing process setup
To set up an appropriate testing process, some prerequisites are necessary. First, there is a need for expertise within the team in the section of software quality and testing. Quality assurance is a separate discipline in software projects requiring an understanding of quality concepts and technical know-how, especially when implementing test automation to support frequent test execution in agile environments. Depending on the project size, there is usually a test manager who is responsible for defining a test strategy and ensuring that quality criteria are met. Furthermore, test engineers conduct tests on software in different ways. Usually, there are black-box, white-box, manual and automated tests undertaken to assess the current state of a software release (Spillner, 2019).
Once there is know-how and a mindset established within the team, the next step is to define a test process. As a starting point, the structure of the ISTQB can be taken to set up a testing process. Each stage of the process involves several tasks and activities. Although the stages are listed sequentially, they will be executed iteratively, and there is also some overlap between the stages. Especially in agile projects, where software is built incrementally, the stages are aligned with the SDLC and conducted iteratively with many iterations. The testing process offers a basic framework which needs to be adapted to the project and software being built (ISTQB, 2021):
Conclusion
Ignoring the importance of software testing can lead to tremendous damages in the aspects of financial loss, reputational damage and even loss of life. Many real-world examples show that the lack of software testing is a severe problem for companies around the globe. Setting up an appropriate and professional software testing process is crucial for project success and a good reputation. Although software testing cannot detect every single bug in a software product, it is a commitment showing that software quality is taken seriously, and that processes and standards are set up to deliver high quality to the customers.
All our quality assurance experts are certified by the ISTQB to apply a standardised and worldwide acknowledged testing approach in your project!
Glossary (following ISTQB-Glossary)
Black-Box Testing: A test technique which is aimed at testing the system without knowing the internal structure of the software
End-to-End Test: Test level which tests business processes from start to end in a production-like environment
ESA: European Space Agency
ISTQB: International Software Testing Quality Board
SDLC: Software Development Life Cycle – a list of activities performed at different stages in software development and the connection between them in a logical and chronological manner
Test Engineer: A person who inspects and reports the quality of a system or product through the entire production cycle
Test Manager: A person who is responsible for managing testing activities, resources and evaluation of a test object
Testing Process: A set of interrelated activities to specify and fulfil given quality criteria
Testware: Work products produced during the test process for use in planning, designing, executing, evaluating and reporting on testing
Unit Test: Test level which focuses on single methods or functions in isolation
White-Box Testing: A test technique which is aimed at testing the system with access to the internal structure of the software
References
Auer, Andrea (2021): Ariane 5. https://www.mathe-museum.uni-passau.de/digitale-exponate-zum-ausprobieren/ariane-5/,
ISTQB (2016): Certified Tester Advanced Level Syllabus Test Automation Engineer. https://istqb-main-web-prod.s3.amazonaws.com/media/documents/ISTQB-CT-TAE_Syllabus_v1.0_2016.pdf,
ISTQB (2021): Certified Tester Foundation Level Syllabus. https://istqb-main-web-prod.s3.amazonaws.com/media/documents/ISTQB-CTFL_Syllabus_2018_v3.1.1.pdf, zuletzt
Ronald, Jasmine (2020): 12 Common Software Testing Misconceptions That We Need to Clear. https://dzone.com/articles/12-common-software-testing-misconceptions-that-we,
Spillner, Andreas (2019): Basiswissen Softwaretest: Aus- und Weiterbildung zum Certified Tester – Foundation Level nach ISTQB®-Standard (iSQI-Reihe). 6. Ed. Heidelberg: dpunkt.
Starbucks (2015): Point of Sale Register Outage Resolved. https://stories.starbucks.com/stories/2015/starbucks-point-of-sale-register-outage-resolved/
Wunderlich-Pfeiffer, Franz (2015): In den Neunzigern stürzte alles ab. O https://www.golem.de/news/softwarefehler-in-der-raumfahrt-in-den-neunzigern-stuerzte-alles-ab-1511-117537.html