Tôi là Tester

Mobile application testing today required a fully sophisticated approaches to shape the desired qualities, ideally the deliverables judgment of qualities depend on test test plan and agreements of the whole team. Unlike web application, mobile application testing put up many challenges that troubleshoot us for a long term:

• Devices diversity: A combination of lots of devices fragment in term of models, hardware and OS
• External device factors:
  • Network
  • Interruption
  • Temperature
• Releases: You can roll back , apply quick patches easily on web due to evolved deployments dedicated for web infrastructure, but not easy for devices. We can’t force users to update the app on the device due to mobile OS.
• OS flavor upgrades: An application can be broken after the device’s OS is upgraded, and it’s harder to validate than web application
• Release Approval: A new version of mobile application need to go through Google Play/ Apple review processes, while mobile application doesn’t.

Combined all of above things lead to testing variations for a specific application need to be applied for smooth release. Remember defects prevention is better than fixing them to reduce the efforts and chaos upon the releases

The better information

The most important of catching root cause issues laid in application/device/custom logs generated from the session. Check and add into the framework the capabilities to collect logs for analytics.

The better strategy

Measure and determine the release’s status on mobile testing application is not laid to functionalities anymore. Only functionalities testing can help aid in determine how the application is performed exactly what we want, not what users want and even how is it performed on different devices. Rather than, inject different kinds of testing ways to the application

Crash Testing

Crash is obviously the most annoying issue. There are many reasons crash can be occurred and quick way to check it is using chaos method applied by randomized user inputs

• Android: Inject monkey runner command
• iOS: Inject randomized user inputs using SwiftMonkey
• Gather crash metrics and analytics with the help of analytics reporting tools

How much of crash rates is enough can be referred in this article. Usually this check is harnessed post-commit and track crash rates along with crash logs

Visual Testing

The more and more devices fragmentation you are going to support, the more difficulties to cover the product look and feel. A normal test script that verify every details will be very exhausting. Visual testing reduce the efforts spent for to adapt smart verification and improve the awareness which the team target on the devices.

Beta Testing

Roll out the RC version to a fixed % of end-users who will help to verify the new features. You still can’t cover all specific device settings and its root cause at the same but this hook greatly improve user experiences. Some platforms help to deploy these are TestFlight or HockeyApp

Performance Testing

This testing is not making performance validations on the server infrastructures sideloaded with the application. It’s about how well the application reacts to the device under test under specific conditions. This can be quite complex, because mobile application performance conclude many different factors:

• Network
• CPU Usage %
• Memory Usage %
• Backend:
  • Data transactions performed in the memory
  • Database queries
• Render:
  • Time until application interaction is allowed
  • Blank/Dark screen availability
• Storage usage

For easy analyze on these kind of factors, personally I recommend to use Apptim which I’ve had an article previously.

Post-release testing

Observations and monitoring post-release is also a critical way to benchmark the release. You probably will want to look at end-user reviews, how the application’s version is being used over time, is the latest version increase the user’s retention with your intuition fixes? Factors count toward the success of the release, and it’s also an indicator for testing to be revamped with more ways of end-users using the release

For this to be success, you need to monitor:

• Reviews on Google Play
• Reviews on Apple Store

Even fewer reviews are on these platforms, the internal tracking of your application will help to determine if the users drop the application or not