Desire high volume app downloads? Fix app-device compatibility issues

Imagine your team is preparing to launch your hot new Android app and you are planning to invest thousands of dollars in an app discovery campaign to get your app into the Android Play Store’s Top Apps list. You are eagerly hoping for large volume of app downloads. And then the mobile app testing issue comes to fore.

Flying blind
As the launch day approaches, it dawns upon your team that the dizzying array of Android phones from multiple vendors has a downside. Your developers inform you that your app does not always behave consistently across all devices. There is a frantic attempt at testing your app on Android devices of as many friends as the developers can persuade. And finally, you take a leap of faith and launch your app with a silent prayer.

Variations in Android devices
The variations in device hardware (the CPU/GPU and memory size, sensors) and OEM customizations to vanilla Android can cause apps to behave quite differently across these devices. Verifying behavior of your app across various device configurations in a systematic manner will help mitigate the risk of consumer complaints after your app hits the Android Play Store.

Inconsistent behavior we have experienced across devices

  • Keyboard pop-up behavior
  • Alignment of controls (Major issue with earlier versions like Android 4.2.2)
  • Material Design Control look and feel in older version (Android)
  • Visibility of various controls changes across devices based zoom level
  • Look and feel of app on Phablets
  • Surprising variance in behavior of native controls (Camera, Audio Records, etc)

And just when you thought there is no hope… TestDroid comes to the rescue
Pray no more. There is comprehensive approach to validate your app behaves consistently across wide range of Android devices — TestDroid service (testdroid.com) allows app developers to install, run and test their app across 350+ unique Android phones (has a wide range of iOS devices as well) attached to their cloud service. These are real devices with Android versions ranging from the latest 6.0 going back all the way to 2.2, on which you can install your app, execute your test scripts and review results and errors logs.

Types of Testing
We have leveraged TestDroid for the following:

  • Functional testing
  • Performance and Stress Tests
  • Regression Testing
  • Interactive Testing

How to automate testing without feeling overwhelmed?
We discovered Appium (appium.io), a terrific open source mobile test automation framework works well on TestDroid cloud with some tweaking. [We found Appium’s thriving open source community helpful in solving issues.] Appium supports a wide variety of programming languages. So no new learning was required — our developers could stay with their beloved programming languages Java and JavaScript.

Some things that worked well for us

  • Developing Appium test cases with every feature
  • Developing test scripts early (Tip: Don’t wait until end of your application development)
  • Running test cases for every app feature on a group of desired mobiles devices on TestDroid
  • Leveraging interactive mode to reproduce device specific bugs reported by test users

Can’t Android emulators suffice?
While emulators are useful testing platforms, testing on them one-at-a-time imposes a big burden. [BTW, Sauce Labs has improved emulator testing by incorporating Android vendor configurations to emulate more realistically and test across many devices. Could be considered as an alternative to TestDroid although we have not tried it yet.]

The bottom line
We have come to the sad conclusion that attempting to launch a mass market app without thoroughly testing for device compatibility is suicidal. The additional effort and cost of testing on TestDroid for app-device compatibility is well worth it — the alternative, fixing bugs reported by users after app is in the store, is expensive. And your app users will thank you for it!