Continuous integration for Android projects

CircleCI is popular among Android developers for several reasons: it’s quick to get started, fast to execute your builds with high parallelism, (whether native, cross- or multi-platform), and even supports running Android emulators right from CircleCI with our Android machine images.

This article will show you how to build and test Android applications for an example project on the CircleCI platform. The full source code is available on GitHub - CIRCLECI-GWP/android-testing-circleci-examples, and you can find the working pipelines in the corresponding CircleCI project.

To get the most out of this article you should have:

Working knowledge of Android development
Familiarity with Android tooling and testing frameworks
Ability to use the Gradle build system

What’s available in CircleCI for Android

There are some useful tools available for building Android:

Docker image
Android machine image
Android orb

Docker image

The Docker image is the main executor teams use. It runs in a Docker container and so is very fast to start, and contains everything you might need to get started building Android projects. The full list of preinstalled tools can be found in the documentation. The images also ship with variants that have Node.js, the Android NDK, and browser tools preinstalled.

Machine image

The biggest change is the new Android machine image. This is a virtual machine image, so takes a bit more time than the Docker image mentioned above, but it contains all the tools you may need to build your Android applications, including the SDK, Google Cloud tools for Firebase, Node.JS, and Fastlane.

The biggest difference is the support for nested virtualisation, which allows you to run Android emulators from within this machine image.

Android machine images support nested virtualization, which allow running the x86 emulators in CircleCI jobs. If you have been developing for Android since the early years of the platform, you may remember that emulators were very slow. Then x86 emulators started supporting Intel’s Hyper-V hypervisors, which lets the emulator use the resources of the host computer directly. Using resource directly like this makes the emulators able to run much, much more smoothly and quickly.

For a long time, support was confined to your local hardware. Limited support on CircleCI slowed down emulators and made it difficult to run UI tests. Nested virtualization solves this problem by adding support for fast emulation to a massively reproducible CI/CD environment, making all the benefits of that available.

Android orb

Finally there is the CircleCI Android orb which provides easy access to both images as executors, as well as handy jobs and commands for installing SDKs, running emulators, caching, testing, and more.

You can read more about what the orb offers in the orb docs at this link.

Introducing the Android CI demo project

I have created a demo project, which is a fork of Google’s own Android Architecture Sample sources. The project is a simple TO-DO list manager application.

Android Todo application

The project contains a combination of unit tests and instrumentation tests, with and without the UI. The project is also available to view on CircleCI. If you open it in Android Studio, make sure to switch to the Project view. Project view lets you review the .circleci/config.yml file.

The project has a single app module, with the default debug and release variants. The test directory contains unit tests, while the androidTest directory contains instrumentation tests. The full CircleCI configuration is included, and I will be referring to it in this article. The configuration is located in .circleci/config.yml.

Building and testing CI for the sample Android app

Your CI process must do three things:

Run unit tests inside a virtual machine
Run instrumentation tests on the emulator
Build a release version of the app

You will run unit tests and instrumentation tests side by side. If both types of tests are successful, you can assemble the release build. You will also add a condition to continue on to the release build only if new work has been commited to the “main” branch, and only following successful emulator tests on most of the major Android versions from 29 to 35. When your build meets this condition, you will have solid assurance that your application will work on many different Android devices.

Note: As I mentioned before, everything from this point on will refer to the CircleCI configuration file in .circleci/config.yml.

Using the CircleCI Android orb

The first step for creating your CI pipeline is to use the Android orb, which contains many of the steps already pre-written for you. The latest version is available in the orb docs.

After the orb is defined, the script specifies these jobs:

unit-test
android-test
release-build

One workflow, test-and-build, is also included.

version: 2.1

orbs:
  android: circleci/android@3.1.0
  gcp-cli: circleci/gcp-cli@3.3.1

jobs:
  unit-test:
    ...
  android-test:
    ...
  release-build:
    ...
    # We'll get to this later

workflows:
  test-and-build:
    ...
  # We'll get to this later

Running unit tests

Your unit-test job contains an executor from the Android orb: android/android_docker. As mentioned this runs it in a Docker container which is extremely fast to start up.

In the steps stanza, there are a few things to note. First, run android/restore_gradle_cache Gradle cache helps store your dependencies and build artifacts. After running the test command, run android/save_gradle_cache to make sure subsequent builds run faster by using the cache. These all come from the Android orb, as indicated by the prefix android/.

You could also modify what gets cached by passing the find-args parameter to restore_gradle_cache. A great advanced example can be found in this open source project.

The main build step happens in the android/run_tests command, where you call ./gradlew testDebug within a test_command parameter. This command runs all unit tests for the debug build variant. As a bonus, this command comes with a default retry value to help you avoid test flakiness.

jobs:
  unit-test:
    executor:
      name: android/android_docker
      tag: 2025.03.1
    steps:
      - checkout
      - android/restore_gradle_cache
      - android/run_tests:
          test_command: ./gradlew testDebug
      - android/save_gradle_cache
      ...

Running Android tests on the emulator

Your android-test job is the one that uses the new Android emulator capability. The job uses the same Android machine executor as the build job. All subsequent jobs will use the same executor, and the cache restoration steps are also the same.

Emulator must run on a machine image, so you specified android/android_machine as the executor. The project uses large machines, but I recommend that you do some experimentation on your own to find the right size executor for your project.

The next step is even shorter; you need just the android/start_emulator_and_run_tests command. This command does exactly what it says: it starts the specified emulator and runs the tests. The test_command is passed again as the parameter (it uses the android/run_tests command). The system_image is the fully qualified Android emulator system image. Your Android machine images are currently bundled back to SDK version 23. You could install more by using the sdkmanager tool.

jobs:
  android-test:
    executor:
      name: android/android-machine
      resource-class: xlarge
    steps:
      - checkout
      - android/start_emulator_and_run_tests:
          test_command: ./gradlew :app:connectedDebugAndroidTest
          system_image: system-images;android-35;google_apis;x86_64
      ...

Note: You could code all the emulator setup steps manually using the commands specified in the Android orb: create_avd, start_emulator, wait_for_emulator, and run_tests.

Storing test results

There is more to testing than just running the tests; you can get a lot of value from seeing exactly what has failed from right inside the CircleCI dashboard. For that you can use store_test_results functions built into the CircleCI platform, which will show your passing (or failing) builds.

The steps differ slightly between unit tests and instrumentation tests, as each of their respective Gradle tasks stores the tests in a different place. For unit tests the tests will be in app/build/test-results, and for instrumentation tests they will be in app/build/outputs/androidTest-results.

We suggest you create a new Save test results step. This step uses the find utility and copies all test results files into a common location - like test-results/junit, and then stores it from there. Make sure to add the when: always parameter to the step, so the step runs regardless of whether the tests above succeed or fail.

Storing unit tests

For this project, you want the XML based results. Calling store_artifacts makes those results parseable on the CircleCI platform. You can also store the HTML output if you prefer.

...
- android/run_tests:
  test_command: ./gradlew testDebug
- android/save_gradle_cache
- run:
    name: Save test results
    command: |
      mkdir -p ~/test-results/junit/
      find . -type f -regex ".*/build/test-results/.*xml" -exec cp {} ~/test-results/junit/ \;
    when: always
- store_test_results:
    path: ~/test-results
- store_artifacts:
    path: ~/test-results/junit

Storing instrumentation tests

There are different regex command for different test types. The rest of the steps to store test results are identical to the unit tests example.

- android/start_emulator_and_run_tests:
    test_command: ./gradlew :app:connectedDebugAndroidTest
    system_image: << parameters.system-image >>
- run:
    name: Save test results
    command: |
      mkdir -p ~/test-results/junit/
      find . -type f -regex ".*/build/outputs/androidTest-results/.*xml" -exec cp {} ~/test-results/junit/ \;
    when: always
- store_test_results:
    path: ~/test-results
- store_artifacts:
    path: ~/test-results/junit

Making a release build and storing the artifacts

For this project, you need to store the application you are building. To complete this part of the process, you’ll use the release-build job, which has two parts of note. A run step calls ./gradlew assembleRelease. Then, store_artifacts points to where the built APK is. For a full CI/CD project, you could sign and upload that APK to a beta distribution service. You could even create a release on Play Store with Fastlane. Both of those activties are out of scope for this article, though.

jobs:
  ...
  release-build:
    executor:
      name: android/android_docker
      tag: 2025.03.1
    steps:
      - checkout
      - android/restore_gradle_cache
      - run:
          name: Assemble release build
          command: |
            ./gradlew assembleRelease
      - store_artifacts:
          path: app/build/outputs/apk/release/app-release-unsigned.apk

Running tests on multiple Android versions simultaneously

The Android platform evolves all the time, and each version has its own bugs and pecularities. I am sure you have encountered some of them. A good way to avoid unexpected behaviour across different versions of Android is to test on as many of them as you can. The CircleCI job matrix feature makes it easier to run jobs on multiple versions.

To use the job matrix, add a parameter to the job you want to test on multiple versions. Your android-test job does that using the system-image parameter. You have also specified a default value for the parameter, so you can run the job without it.

To use the parameter, specify where you need its value by placing it within pairs of angle brackets - << >>:

android-test:
    parameters:
      system-image:
        type: string
        default: system-images;android-35;google_apis;x86_64
    executor:
      name: android/android_machine
      resource_class: large
      tag: 2024.11.1
    steps:
      - checkout
      - android/start_emulator_and_run_tests:
          test_command: ./gradlew :app:connectedDebugAndroidTest
          system_image: << parameters.system-image >>

To pass values to the parameter in the workflows, use the matrix parameter of the job you are calling.

workflows:
  jobs:
  ...
  - android-test:
      matrix:
        alias: android-test-all
        parameters:
          system-image:
            - system-images;android-35;google_apis;x86_64
            - system-images;android-33;google_apis;x86_64
            - system-images;android-32;google_apis;x86_64
            - system-images;android-30;google_apis;x86
            - system-images;android-29;google_apis;x86
      name: android-test-<<matrix.system-image>>

Making each run faster by caching

Alongside building your application, installing dependencies, and running tests, you may want to leverage cache as well. This allows you to do repeated tasks less often. Android and especially Gradle provides an excellent mechanism for this - the Gradle Build cache.

This lets you skip pre-built parts of the application.

Choosing what runs when

During development, you create many commits, and developers should be encouraged to push them to a remote repository as often as possible. Each one of these committs creates a new CI/CD build, though, and you might not need every test run on all possible devices for a single commit. On the other hand, if the main branch is your primary source of truth, you want to run as many tests on that as possible, to ensure it is always working as intended. You might want to initate the release deployment only on main, and not on any other branch.

You can use CircleCI’s requires and filters stanzas to define workflows that fit your process. Your release-build job requires the unit-test jobs. That means the release-build is put in the queue until all the unit-test jobs have passed. Only then is the release-build job run.

You can use filters to set up logical rules for running a particular job only on a specific branch or git tag. For example, you have a filter for your main branch that runs the android-test with a full matrix of emulators. In another example, the release build triggers only on main, and only when its two required jobs have run successfully.

workflows:
  test-and-build:
    jobs:
      - unit-test
      - android/run_ui_tests:
          executor:
            name: android/android_machine
            resource_class: large
            tag: 2024.11.1
          filters:
            branches:
              ignore: main # regular commits
      - android-test:
          matrix:
            alias: android-test-all
            parameters:
              system-image:
                - system-images;android-35;google_apis;x86_64
                - system-images;android-33;google_apis;x86_64
                - system-images;android-32;google_apis;x86_64
                - system-images;android-30;google_apis;x86
                - system-images;android-29;google_apis;x86
          name: android-test-<<matrix.system-image>>
          filters:
            branches:
              only: main # Commits to main branch
      - release-build:
          requires:
            - unit-test
            - android-test-all
          filters:
            branches:
              only: main # Commits to main branch

Running the workflow on CircleCI

On the CircleCI dashboard, click the Projects tab. Search for the GitHub repo name and click Set Up Project for your project.

Setup project on CircleCI

You will be prompted to add a new configuration file manually or use an existing one. Since you have already pushed the required configuration file to the codebase, select the Fastest option and enter the name of the branch hosting your configuration file. Click Set Up Project to continue.

Configure project on CircleCI

Completing the setup will trigger the pipeline and after a few minutes the build should succeed.

Successful build on CircleCI

You can click on android-test-system-images jobs to view details of the instrumentation test execution.

Successful build details on CircleCI

Note that the first execution of the pipeline might take longer to complete depending on the complexity of the Android project. The executor downloads the dependencies, persists the Gradle cache for subsequent runs using android/save_gradle_cache step. In subsequent runs, the executor uses the android/restore_gradle_cache steps to restore the cache, resulting in significantly faster builds, often reducing build times by more than 50%.

If you need even faster builds, consider using a larger resource class (such as xlarge) for your jobs. Some of these resource classes are available on a paid plan, but might be better suited for continuous integration or PR builds.

Improving your Android application CI process

The steps I have described in this article are only a beginning. There are many ways you can improve the flow. Here are just a handful of ideas:

Build once, run many times

If you are an experienced Android developer you know that the :app:connectedAndroidTest will always run the entire build process from the start. Using CircleCI, you could build the entire application and tests once, pass the artifacts down to following jobs, and simply run the tests on the emulators. This process could potentially save several build minutes per job run.

To make this happen, add three commandline steps in each emulator run: install the app, install the instrumentation app, and run the instrumentation tests. For your application, you would enter:

adb install app/build/outputs/apk/debug/app-debug.apk  
adb install app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk  
adb shell am instrument -w com.example.android.architecture.blueprints.reactive.test/androidx.test.runner.AndroidJUnitRunner

It is true that this code sends output to the the command line. That is not the tidy test output that Gradle provides. If you are motivated to make the output look better, this StackOverflow topic might help you.

Running on real devices

Emulators are one tool, but real world devices are often (sadly) much different. There is nothing that can replace a real-world device test. For that you can use a solution offering real devices in the cloud, like Browserstack or Firebase Test Lab.

Deploying to beta testers, and releasing the application

This build and test example project is just one part of a wider CI/CD story. You can take your CI/CD process much further. For example, you can automatically release new versions of your applications right to the Play Store. CircleCI has a handful of guides to help you. This tutorial is a great place to start. The official Fastlane documentation is another helpful resource.

Conclusion

In this article, I described building an Android application with CircleCI, and testing it. For our sample project, we used unit tests and a matrix of Android platform emulators, using the new CircleCI Android orb and machine images. We also touched on how to store and display test results, and how to orchestrate workflows to run tests will not give you the tidy test output that Gradle provides, and all output is on a portion of devices.

Site

Blog