A beginners guide to an effective Android testing strategy

Testing quite often is one of the most neglected aspect in many Software projects, and in Android this has been particulary a common issue since the beginning of the platform.
atsl-logo

Even Google didn’t seem to be making big efforts to encourage testing: Documentation was almost nonexistent, the tools and frameworks were confusing or hard to use and there was almost impossible to find an open source project with good testing implementation.

 

Why testing?

Implement a good suite of different type of tests with a good coverage ratio and maintain it requires time and discipline from the dev team. In some cases the member of the team might not be familiar with the testing frameworks or even never have implemented any  type of test. So what are the benefits of this time and resource investment? Here some of the reasons:

  • Testing allows us to check the correctness of the app features. We should never rely on the QA team to find defects in our code. Is a very common practice to implement some feature, test a couple of use cases, send it to the quality team and wait until they find a few bugs for us and then fix it. This is really a bad practice that a professional developer should never do. We should be confident that the deliverables that we release to QA team shoulders been tested throughly, has a good test code coverage and that our quality colleagues should almost never find a bug.
  • Improves the design and modularity of our code base. If we create a new feature with a TDD approach, we will be building our classes in short iterations, thinking in the interface and driving the implementation with the proper user requirements in mind. This will make our modules and classes small, independent and with one only responsibility unit.download
  • Help us to implement new features or refactor existing one with much more confidence that our changes will not break current working and tested
    functionalities. If every time we add a new piece of code to our project we run our test and all pass, we can be sure that we didn’t implement anything that is affecting and having negative side effects on the existing features.

 

Types of test

In the Android platform, generally we will implement two categories of tests: Unit Tests and Integration Tests.

  • Unit Tests.  We will test small and independent pieces of code. It might be a single class, an interface or just a method. We can consider it as the smallest possible testable piece of code part of an specific independent module. In Android we will use JUnit running either locally on the JVM (prefer this one as fastest method) or in an emulator or real device.
  • Integration Tests. We are testing more than one individual module working together that implements a complete functional part of our system. For example we test the Login feature of our application where different entities works together and and we want to test that the login process is accomplished successfully on different scenarios. In this case in Android  we will implement UI Instrumentation tests with Espresso and Automation tests with UI Automator.

 

Architecting the app for testing

As we mentioned before one of the complains of the Android community was that testing the platform was a difficult task. Apart from the lack of a good testing framework, the developers faced the problems of implementing tests for modules (Activities, Fragments, etc) coupled to multiple Android SDK dependencies.

In order to mitigate this issue, one best and most common solutions is to architect our application using the MVP pattern. With this approach we manage to have a clear separation of concerns of our modules making much more easy to unit test our models and use cases classes without having to mock and get around of all the Android SDK dependencies.

Typically we will have our View layer with the Activity or Fragment that have almost no logic at all. A Presenter acting as a bridge and a Model which will have our use cases and where we will probably focus most of our unit tests.

In a previous I made a basic introduction of the MVP pattern where you can see all these concepts with more detail.

Test Pyramid

Following the TestPyramid concept described by Mike Cohn, Unit Tests should be around test_pyramid60-70% of our test code base and the rest 30% should be implemented as End-to-End tests (Integration, functional, UI tests).

The reason for this is that the end to end tests has a few problems, like you need to wait until the app is deployed and running in a device. Then probably the test will have to deal with things like login, registrations, database access, etc.
Also if you find a bug it will be hard to figure out where exactly the problem is, as it could be in any of the multiple modules and components that are involved in the specific user flow.

On the other hand Unit Tests are fast, reliable and they isolate the test scope to very specific and isolated pieces of the system which makes it a much better way to identify the defects in our system.
If we have build our code base using TDD, we will probably have test covered almost all modules of our application which will give us peace of mind that all parts of the application are being tested and any regression would be detected.

Implementing Unit Tests

First let’s add to the project build.gradle file the required dependencies for Espresso, JUnit, UIAutomator, etc.

    // Dependencies for unit testing
    testCompile 'junit:junit:4.12'
    testCompile 'org.mockito:mockito-all:1.10.19'
    testCompile 'org.hamcrest:hamcrest-all:1.3'

    // Android Testing Support Library's runner and rules
    androidTestCompile 'com.android.support.test:runner:0.3'
    androidTestCompile 'com.android.support.test:rules:0.3'

    // Espresso UI Testing
    androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2'
    androidTestCompile 'com.android.support.test.espresso:espresso-intents:2.2'

    // UIAutomator Testing
    androidTestCompile 'com.android.support.test.uiautomator:uiautomator-v18:2.1.1'
    compile 'javax.inject:javax.inject:1'

As we mention before we have two options to run Unit Tests, one is locally on the JVM and the other is running with UI Instrumentation runner. We will favour the local JVM test as they will compile and run much faster. The downside is that these can’t have any Android SDK dependency, but that should be our goal as we saw before using a MVP pattern.
One way to overcome this issue is providing any Android dependency using a mocking framework like for example Mockito.
This will be fine if we just need to mock a couple of dependencies with not many interactions between our code and the SDK, but if we have more complex interactions and we need to spend hours mocking classes, this would be a signal that we should run those test in a emulator or device using Instrumentation Test.

To add the tests in the project we haver will need two folders, one for each type of test as you can see in the picture below, the Unit Test lives in the folder named “test” and the UI test are located in the folder “androidTest”. Both have to be at the same level that our main folder of the project. This is important because otherwise Android Studio will not be able to identify these classes as tests.

Screen Shot 2016-04-02 at 20.14.14

 

Anatomy of a Unit Test

So now we are ready to add a Unit Test, to do this just add a new class to the Test folders of your project and now we will have to specify which runner we want to use, the size type of the test, etc. Let’s have a look at the example below and we will go through the different parts.

@RunWith(MockitoJUnitRunner.class)
@SmallTest
public class EmailValidationTest {

    String EMAIL_REGEX = SignupPresenterImpl.REGEX_EMAIL;

    @Before
    public void setup() {
        MockitoAnnotations.initMocks(this);
    }

    @Test
    public void givenValidEmail_RegexCheckReturnsValid() throws Exception {
        String validEmail = "david.guerrero@quoders.com";
        Assert.assertTrue(validEmail.matches(EMAIL_REGEX));
    }

    @Test
    public void givenInvalidEmail_MissingAt_RegexCheckReturnsError() throws Exception {
        String invalidEmail = "david.guerreroquoders.com";
        Assert.assertFalse(invalidEmail.matches(EMAIL_REGEX));
    }

    @Test
    public void givenInvalidEmail_MissingHost_RegexCheckReturnsError() throws Exception {
        String invalidEmail = "david.guerrero@quoders";
        Assert.assertFalse(invalidEmail.matches(EMAIL_REGEX));
    }

    @After
    public void tearDown() throws Exception {
    }
}

@RunWith(MockitoJUnitRunner.class)
We specify the runner we want to use, it might be JUnit runner, the Instrumentation, Automation runner or even a custom runner. It depends on how and where we want to run our test.

@SmallTest
This annotation define the type of test we are runing, where “SmallTest” is usually a Unit Test, a “LargeTest” an end to end UI or Automation test and a “MediumTest” is when we are testing more than one module working together, like an Integration Test. See the table below to compare these different types.
Screen Shot 2016-04-03 at 21.19.34

@Before
The method marked with this annotation will be called every time before a test is launched. It useful to initialise and arrange any data needed for the tests.

@Test
This define a test method. It usually should follow the “Arrange-Act-Assert” pattern:
– Arrange all necessary preconditions and inputs.
– Act on the object or method under test.
– Assert that the expected results have occurred.

@After
This method will be execute after each test and we will use it to release resources, close files, etc.

Running the tests

Finally to run the tests we need to select “Unit Test” on the Build Variant menu. After this we can just right click at the Test package folder and then on “Run Tests..” (or Debug or Run with Coverage).

Screen Shot 2016-04-03 at 21.56.15.png

 

Implementing Integration Tests

As we saw before, these type of test should cover around the rest 30 or 20 percent of our test code base. We will use the Instrumentation and/or Automation frameworks that basically allows us to simulate user interactions on the application that will run in a real device or an emulator.

We are going to use Espresso as our Instrumentation test  framework. With it we can click on buttons, enter texts, do scrolls, etc. See below an example of a Sign Up screen tests.

 

@RunWith(AndroidJUnit4.class)
@LargeTest

public class SingupTest {

    @Rule
    public ActivityTestRule<SignupActivity> mSignupActivityRule = new ActivityTestRule<>(SignupActivity.class);

    @Test
    public void testperformValidCompleteSignup() throws Exception {

        //  Fill up new user fields
        onView(withId(R.id.editTextCreateUserName)).perform(typeText("david_" + System.currentTimeMillis()));
        onView(withId(R.id.editTextCreateEmail)).perform(typeText("david.guerrero@quoders.com_" + + System.currentTimeMillis()));
        onView(withId(R.id.editTextCreatePassword)).perform(typeText("password"));
        onView(withId(R.id.editTextPasswordConfirm)).perform(typeText("password"));

        //  Click on Sign Up button
        onView(withId(R.id.editTextPasswordConfirm)).perform(click());

        //  Check progress dialog is showed
        onView(withText(R.string.signup_progress)).check(matches(isDisplayed()));

        //  Assert Home screen has been launched
        onView(withText(R.string.title_activity_home)).check(matches(isDisplayed()));
    }
}

As we can see in the example, we are introducing all the fields needed to perform a new user registration, then clicking on the Sign Up button and assert that the Home Activity is launched correctly after this.
Notice how we have marked the test with the “LargeTest” annotation and we are running it with the “AndroidJUnit4” runner.
To run this test we now need to change to Build Variant to the “Android Instrumentation Test” option and then we just need to choose where to deploy and run the app and the test, either in a real or emulated device.

Summary

So that was a quite brief introduction to a much more complex subject as the testing strategy in Android. There are much more to talk about testing but that will be probably part of further posts.

You can find these examples on my work in progress open source project Treepolis in Github.

As usual please leave any comments, suggestion or corrections. What’s your testing strategy?

Advertisements
Tagged , ,

21 thoughts on “A beginners guide to an effective Android testing strategy

  1. Could you please fix the html entities in your code samples? Beside that: Great article!

  2. Tiziano says:

    Great post here, I really need to get into testing as I’m quite new to it.
    Thanks for sharing! 🙂

  3. Hi, Douglas Here
    Great Article,
    Im in Chrome Mac

    public ActivityTestRule<SignupActivity> mSignupActivityRule = new ActivityTestRule<>(SignupActivity.class);

    Again GREAT ARTICLE!

  4. Hi Dave, nice article,
    there is still a quirk with the email strings on firefox.
    String validEmail = "david.guerrero@quoders.com";

  5. Paolo says:

    Thanks, very usefull overview!

  6. Andrew S. says:

    great article, thanks for posting!

  7. […] Everything you need to know about Android testing. Including the setup, dependencies, folders structure and specific examples of unit tests and integration/UI tests. Lot’s of good stuff here. https://davidguerrerodiaz.wordpress.com/2016/03/04/a-beginners-guide-to-an-effective-android-testing… […]

  8. […] more information about a good testing strategy take a look to my blog post about […]

  9. […] Finally we need a good suite of Unit Test that are going to run every time a new build is launched, that as we mentioned before is going to happen for each push that we make in our repo. We will talk about the testing strategy in further posts but in the mean time you can take a look to one of my previous posts. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: