Turbocharging your Android Gradle builds using the build cache

The Gradle Build Cache is designed to help you save time by reusing outputs produced by previous builds. It works by storing (locally or remotely) build outputs, and allowing builds to fetch these outputs from the cache when it determines that inputs have not changed. The build cache gives you the ability to avoid the redundant work and cost of regenerating time-consuming and expensive processes.

Using the build cache can benefit you by:

• Speeding up developer builds with the local cache
• Sharing results between CI builds
• Accelerating developer builds by reusing CI results
• Combining the remote results with local caching for a compounding effect

The build cache allows you to share and reuse unchanged build and test outputs across the team. This speeds up local and CI builds since cycles are not wasted re-building components that are unaffected by new code changes.

NOTE: Gradle Enterprise Build Cache supports both Gradle and Maven build tool environments.

Prerequisites

To follow this tutorial, a few things are required:

1. Basic understanding of the Gradle build tool
2. Knowledge of how to set up an Android project on CircleCI
3. Knowledge of the Android build process.

Building the app

For the sake of time, we will be using a starter project from the previous tutorial in this series Gradle build scans for Android projects: local and CI builds. Download the starter project here. We will also use Gradle’s build cache documentation.

To get started, click the Run button on Android Studio.

Note: If the app does not run on Android Studio, invalidate the cache and restart by selecting Invalidate cache/Restart from the File menu.

Setting up Gradle build cache

For this tutorial, we are using Gradle version 6.7. This is a change from the previous tutorial, so you will be prompted to update the project files.

First, change the last line in the file gradle/wrapper/gradle-wrapper.properties to:

distributionUrl=https\://services.gradle.org/distributions/gradle-6.7-all.zip

Another change is that we will use com.gradle.enterprise as the plugin ID. This plugin must be applied in the settings.gradle file of the project.

Update the settings.gradle file:

plugins {
    id "com.gradle.enterprise" version "3.4.1"
}
include ':app'

As the last change, remove the plugins block (lines 24-26) in the build.gradle. This format is deprecated in Gradle 6.7.

If you prefer, you can use an alternative setup branch of the project. The changes to the project files outlined above can be seen in this GitHub pull request.

With that out of the way, we can continue.

Enabling Gradle build cache

By default, the build cache is not enabled. You can enable the build cache in a couple of ways:

1. On the command line, run your tasks with the --build-cache flag. Gradle will use the build cache for this build only.

2. Put org.gradle.caching=true in your gradle.properties file. Gradle will attempt to reuse outputs from previous builds for all builds. You can prevent Gradle from reusing outputs for any file by using the --no-build-cache flag.

For this tutorial, we will use the second option. Open the gradle.properties file and a new line:

org.gradle.caching=true

How Gradle build cache works

Let me pause here to give you some context on how the build cache works. It will help with the tutorial, and you can share the information with your team. Gradle supports both a local and a remote build cache. Each can be configured separately. When both build caches are enabled, Gradle tries to load build outputs from the local build cache first. If no build outputs are found, Gradle tries the remote build cache. If outputs are found in the remote cache, they are then stored in the local cache also, so next time they will be found locally.

Gradle has 3 layers of reuse that prevent potentially expensive tasks from being executed unnecessarily. These layers make your builds faster in these 3 target scenarios:

1. In between consecutive runs of a Gradle build by developers, it is common for many things stay the same, with no changes. The Gradle incremental build feature executes only the tasks that have changed since they were last executed.

2. Developers typically maintain many workspaces on many branches to perform logically distinct tasks. The local cache allows outputs to be quickly reused across workspaces and branches without having to transit any networks.

3. CI nodes and developers often run the same tasks with the same set of changes. The remote cache allows outputs to be reused across users and build agents, so that your team never has to build the same thing twice.

NOTE: Gradle stores (“pushes”) build outputs in any build cache that is enabled and has BuildCache.isPush() set to true.

Configuring Gradle build cache

The next step is to configure the build cache by using the Settings.buildCache(org.gradle.api.Action) block of the settings.gradle file. We will start by configuring the local cache, then move onto the remote cache.

Configure the built-in local build cache

The built-in local build cache, DirectoryBuildCache, uses a directory to store build cache artifacts. By default, this directory is stored in the Gradle user home directory, but its location is configurable.

Gradle will periodically clean up the local cache directory by removing entries that have not been used recently.

For more details about the configuration options, refer to the DSL documentation of DirectoryBuildCache.

Add this code snippet to the settings.gradle file:

...

buildCache {
    local {
        enabled = true
        directory = new File(rootDir, 'build-cache')
        removeUnusedEntriesAfterDays = 30
    }
    remote(HttpBuildCache) {
        enabled = false
    }
}

Sync the project.

From the Android Studio toolbar, select Build then Rebuild. This will generate artifacts in the build-cache directory.

Add the build-cache folder to .gitignore. This addition prevents you from committing the artifacts to source control.

Add the following to .gitignore file:

...
# Local build cache
build-cache

Next, run the following sequences of steps on your local machine. These steps will make your build fully cacheable when locally run, no matter where the project is located:

1. Delete the local build cache, which is stored in the build-cache directory
2. Run the ./gradlew clean test Gradle task on the command line
3. Re-run ./gradlew clean test so that it uses the local build cache you generated in the previous step
4. Make sure that both builds succeed and access their build scan links

Click the links to open each build scan.

If this is your first time, you will be prompted to enter your email address so that you can have the build scan link sent to you. Enter your email address, click Go. Check your email for the build scan notice, then click the link to open your build scan.

If you had already activated the build scans, the link from Android Studio will redirect you to a page without the “Activate your build scan” and “Email” steps.

Make sure caching is configured properly by reviewing the Build cache section of the Performance tab.

Note: Notice that the second link in the screenshot has 100% output requested from cache.

From the second build’s build scan, click Timeline. Make sure that no cacheable tasks did any work. To the end of the your build scan URL, add:

/timeline?cacheableFilter=cacheable&outcomeFilter=SUCCESS

You can also observe build tasks that are not yet cacheable. To the end of the your build scan URL, add:

/timeline?cacheableFilter=any_non-cacheable&outcomeFilter=SUCCESS

Using the remote HTTP build cache

Gradle has built-in support for connecting to a remote build cache backend via HTTP. Using the following configuration, the local build cache is used for storing build outputs, while the local and the remote build cache are used for retrieving build outputs.

NOTE: For this implementation, you need a build cache node. If you have not yet created a build cache node, you can use this one.

Update the contents of settings.gradle file:

plugins {
    id "com.gradle.enterprise" version "3.4.1"
}
include ':app'

boolean isCiServer = System.getenv().containsKey("CI")

buildCache {
    local {
        enabled = false
        directory = new File(rootDir, 'build-cache')
        removeUnusedEntriesAfterDays = 30
    }
    remote(HttpBuildCache) {
        url = 'http://34.75.139.200:5071/cache/'
        allowUntrustedServer = true
        enabled = true
        push = !isCiServer
    }
}

This snippet enables our project to load artifacts from HttpBuildCache. Update the url (line 15) to your build cache node link by appending /cache/ to it. Please note the trailing slash. In this case, I have used the URL I shared earlier.

Sync the project:

The remote build cache configuration has several helpful properties:

• url is the location of the shared remote build cache backend
• Use the allowUntrustedServer parameter if you don’t use a self-signed or untrusted certificate.
• enabled activates or disables the remote build cache
• Use push if your continuous integration server populates the remote build cache with clean builds, while developers pull from the remote build cache and push to a local build cache

Run the next sequence of steps on your local machine. Completing these steps will make your build fully cacheable when it is run locally, no matter what the project location is. These steps are similar to what we did earlier for the local cache.

1. Delete the local build cache, which is stored in the build-cache directory
2. Run the ./gradlew clean test Gradle task on the command line
3. Run the ./gradlew clean test Gradle task a second time
4. Make sure both builds succeed and access their build scan links
5. In the build scan, go to the Build cache section on the Performance tab to make sure caching is properly configured

You have now confirmed that the remote build cache is working as expected. Congratulations!

Now, update the setup to make sure that it works as expected for CI builds. Because we have verified that we can push to the remote cache from a project locally, we should restrict such a push to a CI environment.

Setting up for CircleCI

Here is an updated settings.gradle file that demonstrates the recommended setup for the CI push use case:

plugins {
    id "com.gradle.enterprise" version "3.4.1"
}
include ':app'

boolean isCiServer = System.getenv().containsKey("CI")

buildCache {
    local {
        enabled = true
        directory = new File(rootDir, 'build-cache')
        removeUnusedEntriesAfterDays = 30
    }
    remote(HttpBuildCache) {
        url = 'http://34.75.139.200:5071/cache/'
        allowUntrustedServer = true // Allow untrusted cache server
        enabled = true
        push = isCiServer
    }
}

Purge your remote build cache node so that there are no existing artifacts when we check that the CI push mechanism is working.

After you make the code changes, commit them to source control and push to your remote branch.

After a successful CI build, find the build scan on the Circle CI dashboard under the Run Tests job.

Open the build scan in your browser. On the Performance tab of the build scan, review the Build cache section to make sure caching is properly configured.

Here is a key to the items show on the previous screenshot.

• (1) Hits from both the local and remote build cache
• (2) Local cache is enabled
• (3) Local cache path
• (4) Remote cache enabled
• (5) Remote cache URL path

Check the remote build cache node to make sure that the cache artifacts are being saved.

Conclusion

In this tutorial, you learned how to:

• Enable the build cache in our project
• Configure the build cache for local and CI builds.
• Explain how the build cache works to your friends and team members
• View build scan information to confirm that caching is working as expected

And with that, you are ready to meaningfully improve your build performance and increase developer productivity. For the next related tutorial, see Deploying your Gradle Build Cache Node using GCP or Using Gradle build scans in Android projects.