Overview

In this post, we’ll go over each step in building a basic software delivery pipeline. To do this, we’ll provide some examples to demonstrate an automated software delivery process with continuous delivery using CircleCI as a continuous integration server, and a package repository on packagecloud.

To learn more about fundamental concepts around software packages and package management and how to combine them with CI/CD to build a software delivery pipeline, read the previous post Continuous package publishing, part I: introduction to package management in CI/CD.

Building the software delivery pipeline

As an example, we’re going to build a NodeJS package that provides a function that prints ‘Hello World’ and a corresponding test. We’ll be checking code into a git repository, using CircleCI as a continuous integration server, and releasing the package to a repository on packagecloud.io.

Before we start

If you’d like to follow along and create an example software delivery pipeline with the example in this post, please read below to get set up:

  1. Clone the GitHub repo with the example code
  2. Create, or log into an account on CircleCI.
  3. Create, or log into an account on packagecloud and follow the instructions to create a repository. The packagecloud credentials will be used in the example.
  4. Set an environment variable in CircleCI called NPM_TOKEN with the value being the packagecloud user API token.

First, let’s create the package.json file with the metadata for the package we’re constructing.


package.json
{
  "name": "hello-world",
  "version": "0.1.0",
  "description": "Print 'Hello world!'",
  "main": "hello.js",
  "scripts": {
    "test": "jest"
  },
  "devDependencies": {
    "jest": "^22.2.1"
  },
  "author": "Armando Canals",
  "license": "MIT"
}

The main field points to the main entry point to your NodeJS program. In this case, hello.js. And the corresponding program that our package will be containing:

hello.js
function hi() {
  return "Hello world!";
}

module.exports = hi;

As you can see, this program defines a named function that returns a string and then exports the function as a module in a NodeJS application.

The module.exports code is necessary to require the package, as will be demonstrated later. Let’s add a test to cover this functionality. We’ll use the Jest testing library (listed in devDependencies in the package.json) because it’s simple and requires no configuration.

hello.test.js
const hi = require('./hello.js');

describe('hi', () => {
  it('should return Hello world!', () => {
    expect(hi()).toBe("Hello world!");
  });
});

With a test in place, we can ensure that our code is ready to be packaged for distribution.

Configuring CircleCI to run tests

In this next step, we’re going to configure a workflow in CircleCI for running tests and distributing an npm package using the CircleCI 2.0 configuration.

First, in the CircleCI UI, find and click the Add Projects button, select one from the list of available projects, and hit the Start building button.

continuouspackage1.png

Next, in the project root, create a folder called .circleci, and within that folder create a file called config.yml.

.circleci/.config.yml
# Javascript Node CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-javascript/ for more details
#
version: 2

defaults: &defaults
  working_directory: ~/repo
  docker:
    - image: circleci/node:8.9.1

jobs:
  test:
    <<: *defaults  
    steps:
      - checkout

      - restore_cache:
          keys:
          - v1-dependencies-
          - v1-dependencies-

      - run: npm install
      - run:
          name: Run tests
          command: npm test

      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-

      - persist_to_workspace:
          root: ~/repo
          paths: .

workflows:
  version: 2
  only_test:
    jobs:
      - test

This workflow configuration will run the test job, under jobs, on every commit.

To see more details about this configuration, check out Publishing npm packages using CircleCI 2.0.

At this point, we can check in our source code using git.

$ git add .
$ git commit -m 'Update circle config'
$ git push origin feature_branch

continuouspackage2.png

continuouspackage3.png

The testing part of our software delivery pipeline is complete! Now each time a commit is made to the source code repository, the test suite is run to make sure nothing has broken with the recent changes.

Distributing a NodeJS Application using CircleCI 2.0

Now that we have an automated way of running tests for our NodeJS package, we need to deploy the package to an npm registry as releases are made ready for distribution.

First, let’s create an npm registry on packagecloud:

Once logged in to packagecloud, create an npm registry by hitting the “Create a repository” button. The name of this repository, as well as the username on the account, will be used in the CircleCI configuration below.

Next, let’s modify the previous .circleci/config.yml file to include a deploy job under jobs in the configuration:


# Javascript Node CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-javascript/ for more details
#
version: 2

defaults: &defaults
  working_directory: ~/repo
  docker:
    - image: circleci/node:8.9.1

jobs:
  test:
    <<: *defaults  
    steps:
      - checkout

      - restore_cache:
          keys:
          - v1-dependencies-
          - v1-dependencies-

      - run: npm install
      - run:
          name: Run tests
          command: npm test

      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-

      - persist_to_workspace:
          root: ~/repo
          paths: .

  deploy:
    <<: *defaults
    steps:
      - attach_workspace:
          at: ~/repo
      - run:
          name: Set registry URL
          command: npm set registry https://packagecloud.io/armando/node-test-package/npm/
      - run:
          name: Authenticate with registry
          command: echo "//packagecloud.io/armando/node-test-package/npm/:_authToken=$NPM_TOKEN" > ~/repo/.npmrc
      - run:
          name: Publish package
          command: npm publish

workflows:
  version: 2
  test-deploy:
    jobs:
      - test
      - deploy:
          requires:
            - test
          filters:
            tags:
              only: /^v.*/
            branches:
              ignore: /.*/

This workflow configuration will run our tests and deploy our package to the configured npm registry. Replace the packagecloud user and repo with your user/repo on packagecloud.

  • The test job will run npm test each time a commit is made to a branch
  • The deploy job will publish a version to the configured npm registry when a tagged commit containing the v prefix, is pushed to git (i.e.,v0.1.0).

Before we move forward, let’s get the NPM_TOKEN used in the configuration from packagecloud. The token required is the packagecloud API token and can be found by either:

  1. Logging into packagecloud and hitting the “API Token” button on the sidebar.

  2. Or, by using npm login after configuring the packagecloud npm registry on the machine running the npm login command — this could be a local dev machine used to get the credentials. Read more about npm login in the packagecloud docs.

Next, using the token from packagecloud, let’s set an environment variable in CircleCI called NPM_TOKEN. This token will be used to authenticate with the packagecloud npm registry when publishing packages.

continuouspackage4.png

Alternatively, if you prefer to keep your sensitive environment variables checked into git, but encrypted, you can follow the process outlined at circleci/encrypted-files. Triggering a package release

With the CircleCI configuration above, releases can be triggered when a tagged commit containing the v prefix (i.e.,v0.1.0) has been pushed to the git repository.

$ git tag v0.1.0
$ git push origin v0.1.0

continuouspackage5.png

We now have our test suite running on each commit, and releases of our software deployed when we push tags to git.

continuouspackage6.png

Using the deployed NodeJS package

Now that we have our package in a repository, users will need to install the repository where the package is located. Let’s install the package repository where we deployed the v0.1.0 package in our CircleCI workflow.

We can do this by creating a .npmrc file in the home directory, or project root, depending on if you require npm to use this repository system-wide, or specific to a project.

~/.npmrc
registry=https://packagecloud.io/armando/node-test-package/npm/

This repository contains the hello-world application we created earlier. So, users who configure this as an npm registry on their systems can now run the following command:

$ npm install hello-world

This command will install the hello-world package we created and published in the previous section.

$ node
> hello = require('hello-world');
[Function: hi]
> hello()
'Hello world!'

Conclusion

The automation of manual software delivery processes can significantly reduce the software development cycle time. By creating a deployment pipeline, teams can release software in a fast, repeatable and reliable manner.

<This is a guest post written by Armando Canals, co-founder at packagecloud