Florian Gumhold
ERNI Switzerland
Introduction
The testing of software applications is a challenging and demanding task within the software development process. Quality assurance in projects is often given little attention and few resources. This is associated with some risks since unsatisfactory product quality is associated with a negative customer experience in the company’s reputation and sales. Testing should therefore have an adequate status within the software development process. Regardless of the type of software to be developed, it requires a professional and extensive testing strategy.
However, the content and focus of the testing process differ depending on the software application to be tested. For example, testing a freely accessible web application on the Internet requires a different strategy than testing an internal desktop application. Testing mobile applications provides additional challenges because they differ from traditional software applications in many ways. This article describes the challenges that exist and how they can be overcome.
Challenges of mobile app testing
Since the advent of the first smartphones, the importance of mobile applications has increased continuously, and they are an integral part of everyday private and professional life. In contrast to traditional applications on computers, some additional challenges arise for developing and testing mobile applications.
Basically, in the mobile environment, a distinction can be made between native applications and web apps. Native apps are developed specifically for a platform and make optimal use of the functions of the operating system, the hardware and other system functions. In contrast, web apps are suitable for developing platform-independent mobile applications. This results in the advantage that development can be made cheaper, and the mobile web app can be made more widely available on the market since it runs on the smartphone browser without installation.
Furthermore, the diversity of mobile devices is pervasive. Different smartphones with different hardware and operating system versions lead to an almost unlimited landscape of scenarios for mobile applications. The mobile environment also includes additional hardware components that must be considered. The application can address smartphone sensors, the mobile application’s resource consumption is relevant for the device’s battery life, and the mobile data connection is not always guaranteed due to mobility.
These devices’ use in different places in everyday life also challenges the design and usability of mobile applications. Because smartphones cover many use cases, there is the additional fact that the mobile application is embedded in an existing landscape and coexists with many other functionalities. This includes dealing with interruptions caused by calls, the arrival of notifications or the authorisation model for the app.
Based on the business model of the mobile application, free or paid, the first impression often decides whether the user continues using it or uninstalls it. The selection of applications in the stores is large, and the storage space on the smartphone is limited. Finally, with native apps, one is still confronted with the situation that the operating system operators have defined guidelines for approval in the respective stores, which must be fulfilled to offer the mobile application for download.
Strategies for mobile app testing
After the unique features of mobile applications have been explained in detail, the question arises of how this type of application can be tested efficiently and purposefully. These seven approaches help in minimising the risk of insufficient quality and when adapting the test strategy in a mobile environment optimally:
Focus on the most commonly used devices and operating systems
Since mobile applications cannot be tested on every smartphone and operating system, testing should focus on the most frequently used devices and operating systems. For this purpose, the platform operators or other companies regularly offer statistics on market sharing, which can serve as a basis for deciding what devices to test.
Consideration of the frequency of error types and non-functional requirements
The test planning and strategy should be tailored to the frequency of reported functional error types by users. Furthermore, performance and usability tend to be the most frequently criticised non-functional requirements. Here, too, statistics from different providers are available for analysis.
Use of cloud-based test platforms
Due to the high diversity and rapid speed of the mobile market, setting up your own device farm is very expensive and time-consuming. For this reason, there are cloud providers who provide an extensive device landscape by providing countless smartphone types and operating systems. This allows the app to be tested on various devices without buying and operating all of them yourself.
Test automation
Test automation increases test coverage in an agile environment and improves quality. There are many tools on the market for the test automation of mobile applications, which can reduce manual testing activities and use resources more sensibly.
Use of emulators and simulators
Emulators and simulators are particularly suitable for testing functionalities early during development. They are included in the standard development environments and simulate the runtime environment or the hardware. While they cannot entirely replace physical devices, they still provide value for quality assurance during development.
Use of crucial figures in the platform operator’s insights
The two major platform operators, Apple and Android, offer the possibility of generating and evaluating statistics for mobile applications with “Apple app analytics” and the “Google Play Console”. This includes, among other things, the frequency of installations and uninstallations, crashes of your mobile apps concerning the smartphone version and operating system, and the possibility of benchmarking against similar apps.
Conducting different types of tests
The test strategy for testing mobile applications should include numerous test types. These include installation tests, performance tests, usability tests, and field and crowd tests. Only conducting extensive functional tests is not enough, since mobile applications must be tested holistically.
Conclusion
Mobile app testing differs in a few ways from testing other applications. This creates additional challenges that must be taken into account. High quality can be achieved in particular through a professional test strategy, experienced software testers and a focus on the right actions. Because many companies are also pursuing the “mobile first” approach, the number of mobile applications will likely increase further, and Mobile App Testing will play an even more important role than it already does.
