Reconsider the Android emulator for faster testing

author
Doug Stevenson
Director of Developer Relations at Mesmer (mesmerhq.com)
April 21, 2020
Mesmer article

There’s no sugar-coating it — Android UI tests can be slow, especially if you’ve accumulated many of them over time.

So, what do you do when you’ve accumulated so many tests that running the tests themselves slows down your development? The Android emulator has improved a lot over the years, and it’s worth taking a closer look at it.

A brief history of the Android emulator

The Android emulator provided by Google has a rather unpleasant history. For the first few years of its existence, it slow, and fairly buggy. Its sluggish behavior was because of the fact that it emulated an ARM chipset when virtually all development machines were running x86. It also didn’t emulate many features of actual physical devices. So, for serious development, we mostly used actual devices anyway.

Then, in 2013, Genymotion came along with an emulator that actually ran Android on x86 in a virtual machine hosted by VirtualBox, which obviously provided better performance. So many developers switched to Genymotion, with only a few caveats (mostly, price).

Around 2016, Google drastically improved the standard Android emulator to use x86 hardware virtualization, and performance continued to improve over time. It took a while for developers to regain trust in the provided tools, but it easily became the best emulation experience that you can get on a desktop. More recently, 2019 saw a massive increase in performance and features with Project Marble.

Get a more powerful development machine (of course)

In fact, virtualization and hypervisor technology has gotten so good over the years, that it might actually be faster to run the Android emulator on some desktops machines than an actual physical device. Google has guidance on hardware acceleration for the emulator, which I suggest giving a quick read for your development platform.

So, if you want to run your tests on an emulator, it’s definitely worthwhile to consider getting a modern, high-performance machine for development. The speed of the emulator will scale up equally with the speed of your machine’s CPU, and your tests will obviously run faster. If your machine has more than 4 cores, you can also shard your tests to run in parallel, and I’ll talk about that in another post.

Yes, I know — buying better hardware is hard advice to hear for those who can’t afford it! Fortunately, there are more things you can do to boost the performance of your tests that don’t involve spending money.

Disabling animations

One trick you can apply to speed up your tests, regardless of what you use to run the tests, is disabling animations. This is especially important if your tests are heavy on starting activities using ActivityTestRule. Eliminating the cost of animations can shave hundreds of milliseconds off each activity transition. In fact, Activities, in general, are relatively expensive to test, in terms of performance.

You can turn off animations in two ways. The traditional way is to go into the hidden developer settings and manually disable all three types of animations. This is even the first step of Espresso setup instructions.

The linked documentation states that the reason is to avoid test flakiness, but the performance benefits are also real. I’ve even seen several articles that recommend for end users to disable animations to make their devices “feel” faster.

Another, newer way to disable animations is a configuration in your app’s build.gradle. You can set android.testOptions.animationDisabled to true:

android { testOptions { animationsDisabled true } }

With this set, the Android Gradle plugin will automatically arrange for animation to be disabled while running instrumentation tests via Gradle. It does this by passing along the argument --no-window-animation to the test runner command line. (I’ll refer to this command line a few times in this series about Android test performance— it’s a good thing to understand, even if you don’t work with it directly.)

Note that this setting has no effect when running on devices that already have animations disabled in the developer settings. Nor does it help on cloud test services, such as Firebase Test Lab, where animations are already disabled in the same way.

Choose smaller, lower density screens

When configuring an emulator for use with testing, your UI tests will run faster on virtual screens with fewer pixels. The logic here is that pushing fewer pixels automatically translates to higher performance. When you’re choosing the hardware configuration for a new emulator in Android Studio, the UI promotes Pixel-equivalent devices, but if you scroll down, you’ll see options for medium and low density devices with small screens.

Even though real devices with these specs would be fairly low-end devices, running code in an emulator with this configuration doesn’t negatively impact performance at all. The one downside here is that your app’s UI should be prepared to run on such device configurations. Developers often opt not to provide appropriate design and assets for these configurations, so that might be a problem for some apps.

Try the “headless” option

Modern versions of the Android emulator now have a “headless” build. This build will not open up a window when launched. This is normally used when testing as part of continuous integration on virtualized machines that don’t have displays. However, you can also run a headless build on your desktop. Running headless will reduce the amount of memory and CPU required to run the emulator, which could be better used elsewhere on your machine (or to run another emulator, so you can shard tests).

Note that running in headless mode does not arrange for the emulator to pretend there is no display attached to the virtual device. It simply doesn’t make the display visible to you in a window. The emulator is still drawing to a display buffer, and you can still take screenshots and videos of it running as you would normally. You’ll just miss out on seeing the UI of your app while it gets tested.

The documentation and release notes for headless builds talks about a binary called “emulator-headless” in the SDK installation directory. As I write this, using emulator version 30.0.5, I could find no such thing. But you can run in headless mode on the command line using emulator -no-window. If you don’t have experience running the emulator using the CLI, you might want to review the emulator CLI documentation to learn how to use it.

The Android emulator provided with the Android development tools is great, and it keeps getting better. If you’re not using it to the max, it’s worth looking into to speed up your tests and optimize your workflow.

About Mesmer

Mesmer, the leader in Robotics Process Automation for Development (RPAD), is radically changing the way developers work. Mesmer’s AI-powered bots use patent-pending Deep Learning Automation (DLA™) to accelerate every function of customer experience testing. This means means crazy fast releases, better apps, and happier employees. Mesmer is headquartered in Palo Alto, California, and funded by Intel Capital and True Ventures.


Yay, you're on the waitlist!

Skip ahead in line by referring friends using the links below: 


Request Access

Ready to try Mesmer? Request access here, and we’ll get the conversation started.