Overview
In this post, we’ll go over each step in building a basic software delivery pipeline. To do this, we’ll provide some examples to demonstrate an automated software delivery process with continuous delivery using CircleCI as a continuous integration server, and a package repository on packagecloud.
To learn more about fundamental concepts around software packages and package management and how to combine them with CI/CD to build a software delivery pipeline, read the previous post Continuous package publishing, part I: introduction to package management in CI/CD.
Building the software delivery pipeline
As an example, we’re going to build a NodeJS package that provides a function that prints ‘Hello World’ and a corresponding test. We’ll be checking code into a git
repository, using CircleCI as a continuous integration server, and releasing the package to a repository on packagecloud.io.
Before we start
If you’d like to follow along and create an example software delivery pipeline with the example in this post, please read below to get set up:
- Clone the GitHub repo with the example code
- Create, or log into an account on CircleCI.
- Create, or log into an account on packagecloud and follow the instructions to create a repository. The packagecloud credentials will be used in the example.
- Set an environment variable in CircleCI called
NPM_TOKEN
with the value being the packagecloud user API token.
First, let’s create the package.json
file with the metadata for the package we’re constructing.
package.json
{
"name": "hello-world",
"version": "0.1.0",
"description": "Print 'Hello world!'",
"main": "hello.js",
"scripts": {
"test": "jest"
},
"devDependencies": {
"jest": "^22.2.1"
},
"author": "Armando Canals",
"license": "MIT"
}
The main
field points to the main entry point to your NodeJS program. In this case, hello.js
.
And the corresponding program that our package will be containing:
hello.js
function hi() {
return "Hello world!";
}
module.exports = hi;
As you can see, this program defines a named function that returns a string and then exports the function as a module in a NodeJS application.
The module.exports
code is necessary to require the package, as will be demonstrated later.
Let’s add a test to cover this functionality. We’ll use the Jest testing library (listed in devDependencies
in the package.json
) because it’s simple and requires no configuration.
hello.test.js
const hi = require('./hello.js');
describe('hi', () => {
it('should return Hello world!', () => {
expect(hi()).toBe("Hello world!");
});
});
With a test in place, we can ensure that our code is ready to be packaged for distribution.
Configuring CircleCI to run tests
In this next step, we’re going to configure a workflow in CircleCI for running tests and distributing an npm package using the CircleCI 2.0 configuration.
First, in the CircleCI UI, find and click the Add Projects
button, select one from the list of available projects, and hit the Start building
button.
Next, in the project root, create a folder called .circleci,
and within that folder create a file called config.yml
.
.circleci/.config.yml
# Javascript Node CircleCI 2.0 configuration file
#
# Check {{ '/language-javascript' | docs_url }} for more details
#
version: 2
defaults: &defaults
working_directory: ~/repo
docker:
- image: circleci/node:8.9.1
jobs:
test:
<<: *defaults
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
- v1-dependencies-
- run: npm install
- run:
name: Run tests
command: npm test
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package.json" }}
- persist_to_workspace:
root: ~/repo
paths: .
workflows:
version: 2
only_test:
jobs:
- test
This workflow configuration will run the test
job, under jobs
, on every commit.
To see more details about this configuration, check out Publishing npm packages using CircleCI 2.0.
At this point, we can check in our source code using git
.
$ git add .
$ git commit -m 'Update circle config'
$ git push origin feature_branch
The testing part of our software delivery pipeline is complete! Now each time a commit is made to the source code repository, the test suite is run to make sure nothing has broken with the recent changes.
Distributing a NodeJS Application using CircleCI 2.0
Now that we have an automated way of running tests for our NodeJS package, we need to deploy the package to an npm registry as releases are made ready for distribution.
First, let’s create an npm registry on packagecloud:
Once logged in to packagecloud, create an npm registry by hitting the “Create a repository” button. The name of this repository, as well as the username on the account, will be used in the CircleCI configuration below.
Next, let’s modify the previous .circleci/config.yml file to include a deploy job under jobs in the configuration:
# Javascript Node CircleCI 2.0 configuration file
#
# Check {{ '/language-javascript' | docs_url }} for more details
#
version: 2
defaults: &defaults
working_directory: ~/repo
docker:
- image: circleci/node:8.9.1
jobs:
test:
<<: *defaults
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
- v1-dependencies-
- run: npm install
- run:
name: Run tests
command: npm test
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package.json" }}
- persist_to_workspace:
root: ~/repo
paths: .
deploy:
<<: *defaults
steps:
- attach_workspace:
at: ~/repo
- run:
name: Set registry URL
command: npm set registry https://packagecloud.io/armando/node-test-package/npm/
- run:
name: Authenticate with registry
command: echo "//packagecloud.io/armando/node-test-package/npm/:_authToken=$NPM_TOKEN" > ~/repo/.npmrc
- run:
name: Publish package
command: npm publish
workflows:
version: 2
test-deploy:
jobs:
- test
- deploy:
requires:
- test
filters:
tags:
only: /^v.*/
branches:
ignore: /.*/
This workflow configuration will run our tests and deploy our package to the configured npm registry. Replace the packagecloud user and repo with your user/repo on packagecloud.
- The
test
job will runnpm test
each time a commit is made to a branch - The
deploy
job will publish a version to the configured npm registry when a tagged commit containing thev
prefix, is pushed togit
(i.e.,v0.1.0
).
Before we move forward, let’s get the NPM_TOKEN
used in the configuration from packagecloud. The token required is the packagecloud API token and can be found by either:
-
Logging into packagecloud and hitting the “API Token” button on the sidebar.
-
Or, by using
npm login
after configuring the packagecloud npm registry on the machine running thenpm login
command — this could be a local dev machine used to get the credentials. Read more aboutnpm login
in the packagecloud docs.
Next, using the token from packagecloud, let’s set an environment variable in CircleCI called NPM_TOKEN
. This token will be used to authenticate with the packagecloud npm registry when publishing packages.
Alternatively, if you prefer to keep your sensitive environment variables checked into git, but encrypted, you can follow the process outlined at circleci/encrypted-files. Triggering a package release
With the CircleCI configuration above, releases can be triggered when a tagged commit containing the v
prefix (i.e.,v0.1.0
) has been pushed to the git
repository.
$ git tag v0.1.0
$ git push origin v0.1.0
We now have our test suite running on each commit, and releases of our software deployed when we push tags to git.
Using the deployed NodeJS package
Now that we have our package in a repository, users will need to install the repository where the package is located. Let’s install the package repository where we deployed the v0.1.0
package in our CircleCI workflow.
We can do this by creating a .npmrc
file in the home directory, or project root, depending on if you require npm
to use this repository system-wide, or specific to a project.
~/.npmrc
registry=https://packagecloud.io/armando/node-test-package/npm/
This repository contains the hello-world application we created earlier. So, users who configure this as an npm registry on their systems can now run the following command:
$ npm install hello-world
This command will install the hello-world package we created and published in the previous section.
$ node
> hello = require('hello-world');
[Function: hi]
> hello()
'Hello world!'
Conclusion
The automation of manual software delivery processes can significantly reduce the software development cycle time. By creating a deployment pipeline, teams can release software in a fast, repeatable and reliable manner.
<This is a guest post written by Armando Canals, co-founder at packagecloud