 | Issue: | Europe I 2011 |
| Article no.: | 9 | |
| Topic: | Automating mobile application testing | |
| Author: | Eran Yaniv | |
| Title: | CEO | |
| Organisation: | Perfecto Mobile | |
| PDF size: | 360KB |
About author
Eran Yaniv is the CEO of Perfecto Mobile; he came to Perfecto Mobile from Comverse America where he was VP of Product Marketing. Prior to assuming his responsibilities at Comverse, Mr Yaniv was the general manager of the Valisys Quality Management business unit in Tecnomatix. He served as the director of R&D in Tecnomatix Ltd in Israel, where he managed most of the company’s development activities. Eran Yaniv earned a BSC and MSc in computer science from Tel Aviv University.
Article abstract
Smartphones, tablets and such have given birth to a new, enormous, market for applications (apps). Users expect apps to work on any device and network from the start so they must be tested. Testing mobile applications is time-consuming and daunting, but now developers have only to access devices through the cloud and run automated tests. The cloud helps testing from design to pre-launch testing to post-launch monitoring; resulting in faster launches and better mobile applications that cost less to develop.
Full Article
The market for mobile communications has one of the highest levels of mass adoption ever seen; today, mobile devices, smartphones and tablets are omnipresent. The ubiquity of mobile computing presents an opportunity and a challenge for marketers and developers. Users have come to expect a mobile version of nearly every application, and developers are understandably eager to meet those expectations. However, the approaches to mobile application testing upon which companies relied in the past are no longer adequate. Today, the competitive and dynamic mobile market includes a rapidly proliferating number of handsets, tablets, operating platforms and carriers. Developers need to test for all of the many variables that their applications might encounter. This raises numerous questions about testing methods and reliability. There are several factors businesses should keep in mind during the testing phase of application development: In an ideal world, developers would have the time and resources to test every application on a physical device or tablet. This is not an ideal world. Apple gets the credit for breaking ground and building the public’s hunger for apps. Its App Store dominates the market with ten million downloads every day. However, Apple is no longer alone in this market. Operating systems like Android are gaining popularity, enabling third-party handset manufacturers such as HTC and Motorola to create Android-based devices and compete for market share. When we consider tablets, the landscape looks even more crowded. That is both a boon and a difficulty for application developers who want to deliver to every user on every device and on every network. Developers could, once, test their applications on actual devices to find flaws and enhance performance. However, the current diversity of devices, operating systems and carriers makes it virtually impossible to manually test for reasonable quality standards with a sufficient range of physical mobile devices. Attempting to do so poses major time-to-market implications. Physical device-testing is both time-consuming and error prone, often leading to late bug discovery and significant delays in the release of applications. For new companies on a budget, physical testing is especially problematic, since it is difficult for a test manager to accurately estimate the time, cost and resources for the project. Moreover, a testing lab for physical devices is expensive, especially when new devices are launched frequently and must be tested with applications in a multitude of live networks worldwide. There is a time and a place for simulation testing, but it is not adequate on its own. Emulators and simulators give an approximation of devices that can be useful in early stage testing. Their results are close enough to influence preliminary application development. However, when marketers and developers choose to rely solely upon simulators, they often encounter serious flaws when the applications reach the market. Simulation can provide false data related to performance, security and fault-tolerance in various different parts of the world and different networks. The software versions these devices emulate are not always up-to-date, so developers are likely to get a shock when their applications launch after simulation-only testing. There are additional problems with simulation; they don’t offer advanced capabilities like automation, so the testing process can be protracted, delaying efforts to release an application. Given the market’s intense competition, application developers cannot afford to bring late, flawed products to market. Automation is essential to maintaining competitive advantage and producing accurate test results. Today, companies can and should mechanize the tedious, repetitive trial scenarios that are necessary, but time-consuming, to the testing process. Companies can accomplish far more with fewer resources when they have the ability to script multiple usage and test scenarios for data-driven regression and functional testing, schedule them, run them in loops, and run them in multiple devices. Automation transforms tasks that previously took manual testers days to complete. This capability enables the unattended execution of comprehensive automated test scripts on multiple devices. Test automation also has the advantage of accurately documenting the same sequence of steps with each session. This is especially important for bug detection. In manual testing, developers might uncover defects through casual experimentation and then find they are unable to reproduce the steps that produced the bug. Testing scripts can also incorporate built-in functions according to device manufacturer, model, operating system and version, as well as additional elements. The user defines the function once, and then the task becomes part of the automated testing procedure. For instance, users might automate procedures for macro recording, validation and monitoring functions. Functions of this kind make it possible to create simple test scripts in minutes and with minor modifications, reuse the scripts on several devices, and use them again for enhancement and approvals of mobile applications and services. The cloud’s automation tools can reproduce defects and ensure the overall accuracy of regression, integration and application testing, enabling bug detection and correction early in the application development cycle. Finally, developers can use advanced test scripts to help combat the app stores’ complicated submission and certification procedures. All of these factors support the need for automation in mobile application testing. Remote device access lets developers test on multiple devices and carrier networks without depleting resources. We know developers should not rely solely on simulators for application testing. We know that warehousing and manually testing on every single device is impractical, close to impossible, but it is possible to automate the testing for multiple devices with existing technology. We know that automation is essential to a timely application launch. The parameters for exhaustive testing seem to make comprehensive testing impossible, but the introduction of remote device access makes it not only possible to test on multiple devices, but easily accessible and, given market conditions, essential. The cloud can support remote handset testing, troubleshooting and monitoring of applications on real devices over the Internet in real time and across major global mobile networks. Cloud-based platforms provide real devices connected to the Web that can be used to make phone calls, send text messages and download applications. Developers simply choose which device they want from a list of hundreds of options located around the world. The team then remotely runs reliable and automated exercises such as regression testing, functional testing, data-driven testing, security testing, monitoring and more. Given the rapid state of change in the mobile marketplace, remote access testing is a game changer for application developers and marketers. Application testing moves to the cloud Web usage statistics are rising rapidly, and mobile devices are fuelling that increase. As more and more people rely on smartphones and tablet devices for personal and business use, application developers face nearly endless opportunity for innovation. To date, the main hurdles have been their own imaginations and the limitations on testing. The latter is no longer an issue. Testing mobile applications is a time-consuming and daunting process, but now it is a simple matter of accessing devices through the cloud and running automated test scripts. The ´handset cloud’ helps quality assurance teams move from design to pre-launch testing to post-launch monitoring. The results are faster launches and better mobile applications that cost less to develop.
