Android Test Automation Getting to grips with UI Automator

Over the last week I have spent about a day of effort getting to grips with the recently launched UIAutomator test automation framework for Android. It was launched with version 16 of Android (Android 4.1) however on 4.1 devices the framework doesn’t even have all the documented methods available. With version 17 of Android (Android 4.2), support has improved to the point that the examples can work acceptably. Here is the official example http://developer.android.com/tools/testing/testing_ui.html

However in the minor update between Android 4.2.1 and Android 4.2.2 someone seems to have broken the support for automatic scrolling through pages of results.  I have reported the problem on the adt-dev forum, https://groups.google.com/forum/?fromgroups=#!topic/adt-dev/TjeewtpNWf8 which seems to be where the Android development team monitor comments. I have implemented a workaround, using a helper method, below:

    /**
     * Launches an app by it's name. 
     * 
     * @param nameOfAppToLaunch the localized name, an exact match is required to launch it.
     */
    protected static void launchAppCalled(String nameOfAppToLaunch) throws UiObjectNotFoundException {
        UiScrollable appViews = new UiScrollable(new UiSelector().scrollable(true));
          // Set the swiping mode to horizontal (the default is vertical)
          appViews.setAsHorizontalList();
          appViews.scrollToBeginning(10);  // Otherwise the Apps may be on a later page of apps.
          int maxSearchSwipes = appViews.getMaxSearchSwipes();

          UiSelector selector;
          selector = new UiSelector().className(android.widget.TextView.class.getName());
          
          UiObject appToLaunch;
          
          // The following loop is to workaround a bug in Android 4.2.2 which
          // fails to scroll more than once into view.
          for (int i = 0; i < maxSearchSwipes; i++) {

              try {
                  appToLaunch = appViews.getChildByText(selector, nameOfAppToLaunch);
                  if (appToLaunch != null) {
                      // Create a UiSelector to find the Settings app and simulate      
                      // a user click to launch the app.
                      appToLaunch.clickAndWaitForNewWindow();
                      break;
                  }
              } catch (UiObjectNotFoundException e) {
                  System.out.println("Did not find match for " + e.getLocalizedMessage());
              }

              for (int j = 0; j < i; j++) {
                  appViews.scrollForward();
                  System.out.println("scrolling forward 1 page of apps.");
              }
          }
    }

I ended up writing several skeletal demo Android apps to help me explore the capabilities of UI Automator. In each case I was working through publicly reported problems on http://stackoverflow.com where I’ve posted answers and feedback to several reported problems.

Here are the links to my comments:

http://stackoverflow.com/questions/13991977/how-to-switch-on-wifi-in-uiautomator-test-case-in-android-device

http://stackoverflow.com/questions/15204154/uiautomator-failing-on-4-1-2-device

http://stackoverflow.com/questions/15111001/uiautomator-getlasttraversedtext

Strengths of UI Automator

The key strengths include:

  • We can test most applications, including Google’s installed apps such as Settings. Thankfully the example from the Android site does just that, albeit at a perfunctory level. However the example to change the Wi-Fi setting on stackoverflow provides a better example of what we can now do. Because the tests interact with the objects, they have a direct connection to the app being tested, rather than crude interactions by clicking at locations, OCR, etc.
  • Using UI Automator relies on the underlying support for Accessibility in the platform and therefore may help to encourage improved support for Accessible Android apps as developers refine their apps to make them testable by Ui Automator.
  • We can test apps on several devices from one computer, through related changes to the Android build tools.
  • There are debug and exploration tools available on both the device (using adb shell uiautomator) and from my computer, using uiautomationviewer.

Weaknesses

  • Text based matching makes testing localized apps much harder than using the older Android Instrumentation which could easily share resource files with the app being tested.
  • There is virtually no documentation or examples, and the documentation that does exist doesn’t provide enough clues to address key challenges e.g. obtaining the text from WebViews.
  • UI Automation cannot be used when the Accessibility features e.g. Explore-By-Touch is enabled on the device.
  • There are bugs in the current version of Android and there’s no easy way to revert devices to 4.2.1
  • Automation is very slow e.g. paging through the set of apps takes several seconds to go to the next page.

Other characteristics

  • All the tests are bundled into a single jar file, deployed to the device. This risks one set of tests overwriting the bundle of tests.

Further reading

Test Automation Architectures

I recently read a well written and helpful paper written by Doug Hoffman titled: Test Automation Architectures: Planning for Test Automation. You can find it online at http://softwarequalitymethods.com/papers/autoarch.pdf

It covers many key points that need to be considered if you want to have effective and useful automated tests. Thank you Doug for writing it so many years ago and for sharing it.

 

 

Test Automation Interfaces – the glue between your tests and the app

Over the last seven months I have been talking to various people about how test automation ‘works’ and how the working affect the viability of their test automation. In December 2012, LogiGear published an abridged version of an article I have written on the topic http://www.logigear.com/magazine/mobile-testing/test-automation-interfaces-for-mobile-apps/ I hope you find the article informative and helpful.

I sometimes find analogies help people to grasp concepts and ideas which I otherwise might struggle to communicate effectively. So here are a couple of analogies for test automation interfaces:

  1. They are the glue between your automated tests and the app you want to test. By picking the most appropriate glue for the job, your tests are more likely to stick around and work effectively.
  2. The interface is similar to the way Velcro works, the hooks bind with the eyes to establish an effective connection.

I have some ideas and plans to expand the initial article into a small book on effective software test automation. e-mail me if you’d like to encourage that work. My email address is my name (julianharty) at Google’s fine email service: gmail.com I assume a human will be able to create the correct email address from this information 🙂

Slides for presentation at QA&TEST 2011

Here is the link to my slides which I presented at QA&Test 2011. As ever, these slides are the latest version of the work on UX Test Automation.

UX Test Automation for QAandTEST 2011 (27 Oct 2011)

The main content is identical to the presentation planned for EuroSTAR 2011 21st to 24th November 2011. I may revise the main content again by the time of EuroSTAR, if so, I’ll post the updated material online.

Update: I received the best presentation award at the conference for this presentation 🙂

 

 

 

Slides for a talk I presented at Microsoft Redmond

 

Designs and Need adding perspective to our testing (11 Oct 2011) This presentation was given at Microsoft’s Redmond office. The material is not specific to any company or web site, rather I present concepts, ideas, and tools which should be generally relevant to people who want to create software which suits the needs of a wide range of users.

The aim is to encourage additional perspective to our software, as developers, designers and testers, etc. rather than focusing purely on ‘functionality’.

 

 

 

 

 

 

 

Preview of material for StarWest 2011

I hope to meet some of you at StarWest in October where I’m presenting a full day tutorial on testing mobile phone applications on the Tuesday and a track session on pushing the boundaries of test automation on the Wednesday.

If you’d like to come to the conference, the web site is http://www.sqe.com/starwest/

I have made the materials available online and you are welcome to download and use them. The material on ‘pushing the boundaries’ is on this site at UX Test Automation for StarWest 2011 The material for the tutorial is hosted at http://code.google.com/p/mwta/downloads/list

Slides used for STEP-AUTO conference in May 2011

Here are the slides I presented at the STEP-AUTO conference in Bangalore, India, in May 2011. UX Test Automation for STEP-AUTO 2011 (12 May 2011)

I’m continuing to revise the material for various conferences so expect to see updates published on this site from time to time. The next time I present on this topic is at StarWest 2011 Track W9 on Wednesday October 5th. http://www.sqe.com/StarWest/Concurrent/Default.aspx