Mobile app performance Testing with Apptim

Recently I’ve found a tool called APPTIM. This tool is a nice addition to your performance testing on mobile devices providing a bunch of useful features to help visual performance metrics as well as collecting related logs.

This blog is a quick starter guide to help you jump in this tool. I won’t talk about fancy things regards to how important of performance testing, why performance testing is needed etc

To get started and record the session, refer to this guide

The performance bottlenecks

Upon a session is ended, you are presented into a summary screen to have a quick brief about how the application performed on the specific device. Noticable summary information are:

• % CPU Usage
• Avg Memory Usage
• % Memory Usage
• Power Usage
• Downloaded Data
• Errors/Exceptions if any

Visualize performance metrics

Performance testing artifacts always end with of visualized metrics. APPTIM support many common metrics which is parsed from based on dumpsys command. APPTIM actually aggregates ALL the data and round up into a friendly visualization.

• CPU

• Threads:

• Memory:

• Database:

Besides database visualize metric, APPTIM also list out the transaction performed internally in the database’s application. It also points out the WORST transaction (most spent time in ms) to perform

• Power Usage:

All above metrics are parsed from dumpsys output command, which is nice because you will envision yourself into friendly metrics rather than a bunch of raw numbers

Logs

As said from the beginning, all logs including crash, power usage, memory dump, network…. (well so many) are also collected and presented in the local report. Those are all raw logs and bestow for you to dig in more information if you need to debug performance bottlenecks if you can’t seem to figure out from the metrics

—————————————————————————————————————————————————————–

Above are common features for a performance testing tool to record metrics and logs related. BUT APPTIM does more than that:

• Provide activities and layout correctness (Android only): How activities and layouts are used in term of performance determinations. They help mainly developers to determine whether the application need to be adjusted to improve performance as activities and layout is important factors
• Integrate with Jira: You can gather a piece of information from the session and send them directly to JIRA.
• Support exploratory testing: The pros of APPTIM is logs , metrics and misc information collection. That’s why APPTIM supports the ability to take note during the session so that if any bugs found you will have all the needed pieces.
• Support BOT testing: I think it’s coming soon.
• Collaboration: The generated report is also pushed to your workspace. Workspace in APPTIM is your own private page which you can take part in to view the all of previously reports, not just a current one. With paid subscription, collaboration happens by inviting more of your colleagues to

Automation Testing
So does APPTIM support using it for automation testing? So far as I’ve tried, the supports of automation testing tied to local devices and AWS farm.

• Local devices & emulators: Once your session is started, you just need to execute your tests. APPTIM collect performance information, so the automation session works along with this nicely. At the end of automation session, APPTIM also ends it session and performance testing will be collected
• AWS farm devices: Blazemeter conducts an article regards to using APPTIM directly in their Taurus scripts, you can refer here

I will contact more with APPTIM regards to their automation support and APPTIM native scripts and write another article later about this

Summary:

Pros:

• A all-in-one performance collection
• Required dependencies are installed automatically
• Support collaborations through public APPTIM cloud
• Compare two sessions (paid)
• Support JIRA integration
• Provide useful correctness regards to activities and layouts on Android
• All performance, logs, device and extensive information are delivered to you
• Support automation testing

Cons:

• Automation testing support still limited
• The workflow to really start the session is cumbersome a bit

So far I’m liking this tool. It’s a nice addition to help you to start performance testing locally on the devices.