By Florian Gumhold (ERNI Switzerland) and Ashwini Bhat (ERNI Germany)
Software testing should be anchored in every project in order to develop a high-quality and reliable software product. A professional software testing process is a commitment by the company to take quality aspects seriously and to work by defined guidelines and principles to deliver a functional, stable and secure software system.
Medical employees are obligated to complete a medical degree to obtain professional skills in the field of healthcare. These experts need to rely on software systems to apply their expertise in medical facilities and to support the best treatment of patients. A lack of quality assurance can lead to the danger of death, and therefore, quality criteria are not only set by vendors but also by regulating authorities.
As there are many guidelines to be followed to achieve approval of a medical device, the usage of test automation is beneficial to guarantee comprehensive, consistent and documented software testing. With test automation, we can combine functional knowledge with technical expertise, which allows us to run more frequent test runs faster to improve test coverage. Having a framework to automate is not enough here; we need to understand the functional process to have a good set of automation tests.
Regulations in healthcare
Due to the high level of risk in the healthcare industry, the software development and testing process is regulated by authorities. Any software developed or tested needs to comply with ISO 13485 and IEC 62304. Medical devices are used for different purposes in the whole patient care process. Software systems are responsible for creating and collecting relevant data, consolidating and displaying information or even keeping patients alive. The risk of malfunction and unreliability is mitigated by complying with regulations. The FDA (Food and Drug Administration) is one of the biggest regulating authorities in the world and is responsible for the approval of medical devices in the USA. The verification and validation of software is essential when it comes to fulfilling high-level quality criteria. Verification and validation basically describes a documented process which aims to prove that a software system has been built and can be installed correctly and conforms with the expectations of the end user and the requirements defined upfront. The FDA does not dictate how the verification and validation process needs to be performed in detail, but the organisation provides a general guidance document with principles of software validation to fulfil the rules of approval.
The EMA (European Medicines Agency) is responsible for approving medical devices in Europe. Since the FDA is the most restrictive authority, it is common to approve new medical devices in the USA first and then in other countries. This approach subsequently leads to a higher certainty for companies to have their medical devices allowed on other markets too.
The FDA defines two requirements which need to be fulfilled to achieve approval:
- The developed product and processes used need to fulfil the standards of the FDA.
- Every step in the validation and verification process needs to be documented.
Challenges and types of testing processes
Based on almost 30 years of experience in the MedTech industry, at ERNI we have faced the following challenges in Medical Device Testing:
- Device compatibility – With the need to have data available on the move in our everyday life, we need medical devices and apps to be compatible with the web and mobile devices. It is critical that these applications are accessible and safe with all necessary updates.
- Security breaches – Security is one of the most critical aspects of medical devices. To avoid any security breaches, it is necessary to incorporate security tests in every stage of development.
- Cloud adoption – For business development, migrating data to the cloud is very important, which raises concerns about network and data security. To overcome this, we need to adopt cloud application testing and security testing practices during data migration to ensure a more secure network and the safety of cloud data.
To overcome these challenges, especially in healthcare, a professional software testing process is required to fulfil regulations and high-level quality criteria. The software testing process used needs to be traceable and documented. Besides the functional testing of requirements, the following test types need to be performed:
Usability and user experience testing
This testing verifies that the user interface is intuitive and easy to understand and that each element on the screen does what it is intended to do. Problems and complexity in the user interface can lead to confusion and wrong inputs, which could have fatal consequences.
Performance testing
Performance tests guarantee that the medical device can work properly under stress and in high-load scenarios. The software system is not allowed to decrease performance or fail in specific high-load situations. Otherwise, this would lead to the danger of death.
Conformity testing
Regulating authorities have defined production criteria that must be fulfilled to assure that medical devices are fully secure and reliable. Tests against these regulations ensure that approval is granted and maximum reliability is achieved.
Interoperability testing
Medical devices are intended to work in embedded environments and to exchange data and information with other systems. Interoperability tests ensure that medical devices can communicate and interact with these software systems.
Reliability testing
Reliability defines the ability to work on a high level over a long period of time. These tests are intended to prove that the medical device works reliably in every situation, even after a long run time.
Data-driven testing
Manual testing for a huge data set is time consuming. Hence, with test automation, we can create large sets of data and test them in multiple executions.
Authenticated validation
Test automation allows setting a large set of valid and invalid test data to ensure that the device application’s user authentication features are functional.
Test automation
Testing medical devices is a responsible task which requires more attentiveness than in other industries. There is no excuse for malfunction, outage or performance issues. To develop a compliant software system, the testing process should be supported by test automation. Automating tests supports faster test execution and higher test coverage while eliminating human error. In addition, automated tests are documented by generated execution reports, reducing costs. This increase in efficiency results in more time for manual testers to find errors based on their experience and intuition by performing exploratory testing and other meaningful tasks. However, setting up a test automation framework requires expertise and technical know-how. A test automation strategy must be established to define the scope, architecture and test environment. Medical device testing can take advantage of the full capacity of test automation. Not only can functional testing be automated, but performance and reliability testing can also be conducted to prove the conformance of non-functional requirements.
Conclusion
Conducting software tests in healthcare is even more important than in other industries as it is necessary to ensure optimal device performance without major risks. Software systems need to be tested extensively according to guidelines and regulations. A standardised and professional software testing process is the basis for fulfilling given regulations. Due to the comprehensive testing scope and different types of risks, test automation is essential to support test activities. The approval of medical devices does not depend solely on developing software but even more on testing it appropriately. Test automation improves accuracy and test coverage, which in turn helps companies to ensure the dependability, safety and effectiveness of the device in use.