Continuous deployment for Android libraries to Maven Central with Gradle
Developer Advocate
This article will take you through setting up CI/CD integration for building, testing, and publishing libraries to Maven Central using Gradle. With jCenter shutting down, Maven Central is once again the primary destination for all Android and Java libraries. Library publishers will need to port their libraries over to Maven Central to keep their libraries available after jCenter shuts down.
This article focuses on CI/CD integration. Setting up the library itself for manual publishing is out of scope, but you will find some links to useful articles and guides about getting started.
Prerequisites
You will need a few things to get the most from this article:
- Some experience with Android or Java
- Experience with Gradle
- Familiarity with the Java or Android library publishing process
- A CircleCI account
About the project
The sample library is a testing utility of mine released way back in 2016. The relevant parts are all in the Gradle build scripts and the CircleCI config: .circleci/config.yaml.
There are two modules:
- The library module
scrollableScrollis the library being published - The Android app module
sampleis the app we are using to test the library
Note: Testing a library on its own is somewhat difficult.
Publishing libraries to Maven Central with Gradle
If you have not published libraries with Gradle to Maven Central before, this section is for you. If you have done this, you can skip to the next section.
Maven Central or as it is also known, Sonatype OSSRH (OSS Repository Hosting) provides an official guide.
The process requires several steps:
- Register a
groupto publish under. This can be your package name; for my project I usedcom.zmarkan. You will need to create a Jira account and open a ticket for Sonatype Nexus to create a group for you. You will publish all your projects under this group. Note that if you usecom.[domain name]nomenclature you might be asked to prove that you own the domain. - Prepare your library for publishing and signing. Gradle has
maven-publishandsigningplugins available to help with that. There are several articles that cover this process. There are also offical Gradle guides - for publishing, and for signing. - Create a GPG key. This is what you will use to sign the packages.
You can find a complete example in my sample project on GitHub.
If you have everything set up manually, you should get ./gradlew assemble publish to work, which means you are all set.
The manual release process also requires you to log in to Maven Central on the web, and you will need to manually close and release the staging repository that was created during the Gradle publish task.
Continuous deployment of libraries with Gradle
The process described in the previous section would, of course, work for a local publish. In the CI/CD environment this will be slightly different. Firstly, we do not want to perform all these manual tasks. I have used the Gradle Nexus Publish plugin instead, which builds on the maven-publish configuration and automates the last part of the release.
Secondly, we need to make sure that all keys are safely stored and accessible during the CI/CD process for signing. The correct versioning must also be put into place.
Automating releases using the Gradle Nexus Publish Plugin
This Gradle plugin offers two new Gradle tasks to use instead of the existing publish Gradle task:
publishToSonatypecloseAndReleaseSonatypeStagingRepository
To include the plugin, add it to the project level build.gradle file:
plugins {
id "io.github.gradle-nexus.publish-plugin" version "1.0.0"
}
group = 'com.zmarkan' //Or whatever your Nexus publishing group is
version = findProperty('LIBRARY_VERSION')
// Rest of the setup ...
nexusPublishing {
repositories {
sonatype {
//only for users registered in Sonatype after 24 Feb 2021
nexusUrl.set(uri("https://s01.oss.sonatype.org/service/local/"))
snapshotRepositoryUrl.set(uri("https://s01.oss.sonatype.org/content/repositories/snapshots/"))
username = findProperty('OSSRH_USERNAME')
password = findProperty('OSSRH_PASSWORD')
}
}
}
In the example, the username and password values are my Sonatype OSSRH credentials, the same as for the web interface and Jira issue tracker. You can also create separate tokens in Sonatype OSSRH web UI if you don’t want to use your own credentials.
You need to store the credentials in your CircleCI environment variables under the names ORG_GRADLE_PROJECT_OSSRH_USERNAME and ORG_GRADLE_PROJECT_OSSRH_PASSWORD.
Note: Make sure to keep the ORG_GRADLE_PROJECT_ prefixes. Without the prefixes, lookup by findProperty in Gradle will not work.
Deployment flow
The CI/CD pipeline for a library does a few things:
- Tests the library on every commit. The library tests are run on a corresponding sample application. This catches any commits that would break the build as soon as possible
- Releases a snapshot from the
mainbranch. This happens every day at midnight UTC. It allows testing with the most recent version by pointing to the snapshots repository - Releases a new version on every tag. This is the versions most end users will interact with when they use the library
Setting up a CI/CD pipeline for releasing your libraries means your publishing process will still always work the same, no matter what happens to your developer machine, or who is doing the development. You will also keep safe any credentials or signing keys.
Setting up the signing keys is the most involved part of the process, so we should get started.
Signing keys
Signing projects in a CI/CD environment requires a signing key you can pass in to Gradle at build time. To do that you need to export the key in string format, and store both the key itself and its passphrase in CircleCI’s environment variables. Finally, you need to inject the key itself in the project’s gradle.properties at build time.
This means that in your library module’s build.gradle, you can use the useInMemoryPgpKeys function instead. This function takes a string-based signingKey instead of a keyring file. Both keys can be stored as Gradle properties or environment variables:
def signingKey = findProperty('SIGNING_KEY')
def signingKeyPwd = findProperty('SIGNING_KEY_PWD')
signing {
useInMemoryPgpKeys(signingKey, signingKeyPwd)
sign publishing.publications
}
When you run the command to export the signing key, replace your_email@example.com with the email used to generate the key. You will be asked to provide the password you used when creating that signing key.
gpg --armor --export-secret-keys zan@circleci.com \
| awk -v ORS='\\n' '1' \
| pbcopy
The first part of this command gpg --armor --export-secret-keys exports your key into an ASCII string. The awk command replaces any newline characters with \n, and the pbcopy stores it in your clipboard for pasting.
The exported key looks like this (note the plentiful lines of gibberish in the middle):
-----BEGIN PGP PRIVATE KEY BLOCK-----
[hundreds of lines of crypto gibberish]
-----END PGP PRIVATE KEY BLOCK-----
You can then paste it when creating a new CircleCI signing variable GPG_SIGNING_KEY in the project’s environment settings:
You also need to add the password for that key as an environment variable you will use for signing: ORG_GRADLE_PROJECT_SIGNING_KEY_PWD.
Note that the ORG_GRADLE_PROJECT_ prefix is required for the password if you want to use it directly in Gradle with findProperty.
As for the key itself, I found that Gradle was not happy with it being stored as environment variable. I had to write it into gradle.properties before build time with this run call in .circleci/config.yml:
- run:
name: Inject Maven signing key
command: |
echo $GPG_SIGNING_KEY \
| awk 'NR == 1 { print "SIGNING_KEY=" } 1' ORS='\\n' \
>> gradle.properties
Versioning the library with Git tags
All projects in Maven Central need to be properly versioned. I like using Git tags to drive versioning, as we can create them from the command line and they are always tied to a commit.
For any tagged commit, CircleCI will surface the tag as the environment variable $CIRCLE_TAG. I used that tag to denote a release build’s version. For all other commits that are missing tags, I treat them as snapshot builds.
We will store it as another environment variable $ORG_GRADLE_PROJECT_LIBRARY_VERSION that can be picked up in Gradle.
The way to make it available is to push any variables to $BASH_ENV in CircleCI’s run step.
This is the script I used. It either sets the current Git tag as an environment variable (for example 1.0.21), or if it is missing, uses the most recently used tag with -SNAPSHOT appended to it.
- run:
name: Define ORG_GRADLE_PROJECT_LIBRARY_VERSION Environment Variable at Runtime
command: |
if [ $CIRCLE_TAG ]
then
echo 'export ORG_GRADLE_PROJECT_LIBRARY_VERSION=$CIRCLE_TAG' >> $BASH_ENV
else
echo "export ORG_GRADLE_PROJECT_LIBRARY_VERSION=`git tag | tail -1`-SNAPSHOT" >> $BASH_ENV
fi
source $BASH_ENV
Publishing the artifacts to Sonatype OSSRH
The final step of publishing is to run the two Gradle publishing commands right after running assemble:
publishToSonatypecloseAndReleaseSonatypeStagingRepository
This deploys the library and all its artifacts using the Sonatype Staging plugin, and also releases it into production.
- run:
name: Publish to Maven Central
command: ./gradlew assemble publishToSonatype closeAndReleaseSonatypeStagingRepository
The full deployment job as in my .circleci/config.yml:
jobs:
...
deploy-to-sonatype:
executor:
name: android/android-machine
resource-class: xlarge
steps:
- checkout
- run:
name: Define ORG_GRADLE_PROJECT_LIBRARY_VERSION Environment Variable at Runtime
command: |
if [ $CIRCLE_TAG ]
then
echo 'export ORG_GRADLE_PROJECT_LIBRARY_VERSION=$CIRCLE_TAG' >> $BASH_ENV
else
echo "export ORG_GRADLE_PROJECT_LIBRARY_VERSION=`git tag | tail -1`-SNAPSHOT" >> $BASH_ENV
fi
source $BASH_ENV
- run:
name: Inject Maven signing key
command: |
echo $GPG_SIGNING_KEY \
| awk 'NR == 1 { print "SIGNING_KEY=" } 1' ORS='\\n' \
>> gradle.properties
- run:
name: Publish to Maven
command: ./gradlew assemble publishToSonatype closeAndReleaseSonatypeStagingRepository
For CircleCI to run this job you need to do include it in a workflow.
Publishing a release version
My release workflow looks like this:
workflows:
build-test-deploy:
jobs:
- android/run-ui-tests:
name: build-and-test
system-image: system-images;android-23;google_apis;x86
test-command: ./gradlew assemble sample:connectedDebugAndroidTest
filters:
tags:
only: /^[0-9]+.*/
- hold-for-approval:
type: approval
requires:
- build-and-test
filters:
tags:
only: /^[0-9]+.*/
branches:
ignore: /.*/
- deploy-to-sonatype:
name: Deploy to Maven Central
requires:
- hold-for-approval
filters:
tags:
only: /^[0-9]+.*/
It is best practice to set up the tests to run on every single commit, on any branch, but to proceed with deployment only when the commit is tagged like a version. That is what the filters stanza allows. In this case the deployment job only runs on versioned tags that begin with number and a period, like 1.something. That way you can still use tags for other purposes, but they will not trigger a release build.
filters:
tags:
only: /^[0-9]+.*/
branches:
ignore: /.*/
I also added a hold-for-approval, which is an approval job that requires a manual confirmation before deploying the library. This is optional, but I find the manual step reassuring.
Publishing snapshots
Snapshots are special types of releases that are faster to publish, and can be easily overridden, unlike regular, versioned releases, which cannot be overridden. This makes snapshots useful for testing new versions of the library.
To release a snapshot all you need to do is append -SNAPSHOT to the end of the version string. That procedure is covered in the versioning section of this article. I have set my snapshot deployment pipeline so that the library is built each night at midnight UTC, using CircleCI’s scheduled triggers feature:
nightly-snapshot:
triggers: #use the triggers key to indicate a scheduled build
- schedule:
cron: "0 0 * * *" # use cron syntax to set the schedule
filters:
branches:
only:
- main
jobs:
- android/run-ui-tests:
name: build-and-test
system-image: system-images;android-23;google_apis;x86
test-command: ./gradlew assemblesample:connectedDebugAndroidTest
- deploy-to-sonatype:
name: Deploy Snapshot to Sonatype
requires:
- build-and-test
Conclusion
In this article, I covered how to set up a continuous deployment pipeline for an Android or Java library to Maven Central with Gradle. You have learned how to set up the secrets for signing the library, version the library using Git tags, and automatically publish the library each time a new tag is published. I hope you find this useful for your own projects.
If you have any feedback or suggestions for what topics I should cover next, contact me on Twitter - @zmarkan.
