When it comes to deploying features to production servers, most organizations operate using very strict guidelines. Many times, deployments to production require an approval step from a team lead or engineering manager. In this tutorial, I’ll show you how to deploy an API to both a staging and a production environment, authorized by an admin approval. We will create a workflow that deploys the API to the staging environment, then runs automated tests using the Postman CLI tool, Newman. If the tests pass, the API is ready to be deployed. After the admin provides the approval, a deployment job runs to deploy the API to production. If the admin does not provide approval, the API will not be deployed to production.

Prerequisites

To complete this tutorial, you will need:

  1. Node.js installed on your system
  2. Postman for Desktop installed on your system (you can download it here)
  3. A Heroku account
  4. A CircleCI account
  5. A GitHub account

When these items are installed and set up, we can begin the tutorial.

Cloning the API project

To begin, you will need to clone the API project. We will be working with a simple Node.js API application with a root endpoint and two other endpoints for creating and fetching users. Clone the project by running this command:

git clone --single-branch --branch base-project https://github.com/CIRCLECI-GWP/deploy-api.git

When the cloning process is complete, go to the root of the project and install the dependencies:

cd deploy-api
npm install

Next, run the application:

npm start

The application will start listening on a default port of 3000.

Open Postman and make a GET request to the http://localhost:3000/users/get endpoint. This should return an array of users.

Get Users - Postman

Our API is ready to deploy.

Setting up staging and deploy environments on Heroku

The next task is to create the deployment environments. You will be creating one for staging (where the application’s updates will be deployed, for testing) and deploy (the production environment). We will create these environments on Heroku for this tutorial, but you can use the hosting platform you prefer.

Go to your Heroku account dashboard and click New, then Create new app. On the app creation page, create the staging application environment.

Staging App - Heroku

Now, repeat the same process to create the production app.

Production App - Heroku

The environments are now set up to for deploying the application. Finally, on Heroku, get your API key from the Account tab on the Account Settings page.

Setting up API tests with Postman Collections

After it is deployed to the staging environment, we want the API to be tested. For this tutorial, we will use Postman to set up API tests that can be automated.

The first step is to create a dedicated environment for the API requests using Postman.

From the Postman desktop, click the Manage Environments cog icon (it is at the top right corner). The Manage Environments dialog shows any environments that are already in place. Click Add to create a new environment.

In the new environment dialog, enter a name for the new environment. Fill in the staging environment API base URL (https://users-api-staging.herokuapp.com) as an environment variable (api_url). What you enter in INITIAL VALUE is duplicated for CURRENT VALUE. Keep it like that; using CURRENT VALUE is beyond the scope of this tutorial.

Create environment - Postman

Click Add to finish creating the environment. Switch to the new environment using the dropdown at the top right of your screen.

Create a collection

The next step is to create a Postman Collection for the user endpoints of the API you will be testing.

Click Collections from the left sidebar. Then click New Collection.

On the New Collection dialog, fill in the name (Users) of your collection. You can also add a description if you want to add more information about the collection.

Create Collection - Postman

Click Create to finish setting up the collection. The new collection is immediately shown on the left sidebar under Collections.

Adding Requests to the Collection

Now it is time to add requests to your collection. Our API consists of two endpoints:

  • {{api_url}}/users/get (GET): Fetches a list of user profiles
  • {{api_url}}/users/create (POST): Creates a new user profile

You will be adding a request for each endpoint. To get started, click the flyout menu beside the collection (the arrow icon).

Click Add requests. On the New Request dialog, create a request for the {{api_url}}/users/get endpoint. Click Save to Users-API-Collection to save it to the collection you created earlier.

Add Request - Postman

Now that the request is created, a new request tab is loaded. Using the api_url variable you created, enter the endpoint for the request in the address bar ({{api_url}}/users/get). Make sure to select GET as the request method. Click Save.

Next, create a request for the {{api_url}}/users/create endpoint. We need to get random values so that the POST request can create test users. For that, we need a Pre-request script.

Open the Pre-request Script tab and add the following script:

let random = +new Date();

pm.globals.set("name", `Test-User-${random}`);

This script uses the current timestamp to randomly create names for each fired request. The random name variable is set as a global variable for the request instance.

Now write the request body with the dynamic name variable.

Dynamic parameters - Postman

A dynamic name parameter will be used for each request to the /users/create endpoint.

Adding tests

It is time to add some tests to the requests you just created.

Click the {{api_url}}/users/get request (in the left sidebar) to make sure it is loaded. Click the Tests tab. In the window, add:

pm.test("Request is successful with a status code of 200", function () {
  pm.response.to.have.status(200);
});

pm.test("Check that it returns an array", function () {
  var jsonData = pm.response.json();
  pm.expect(jsonData).to.be.an("array");
});

The first test checks that the request returns successfully with a status code of 200. The second test makes sure that it also returns an array.

Click the user creation request Test tab to add:

pm.test("User creation was successful", function () {
  pm.expect(pm.response.code).to.be.oneOf([200, 201, 202]);
});

pm.test("Confirm response message", function () {
  var jsonData = pm.response.json();
  pm.expect(jsonData.message).to.eql("User successfully registered");
});

The first test checks that user creation is successful by asserting a status code of either 200, 201 or 202. The second test makes sure that the correct response message is returned.

Note: Be sure to click Save whenever you make a change to either of the requests in this collection.i>

Configuring automated testing with Newman

Next, we will use Postman’s CLI tool, Newman, to automate how the tests run.

First, your collection will need a public URL. This link will point to the version of your collection hosted on the Postman service. The major advantage of using this link instead of exporting your collection as json is that changes to your collection will always be available using the link as long as you’re logged into Postman. When you sign in on the desktop, Postman synchs all your local collections to your Postman account. If you need a Postman account, the sign-in pagewill guide you through creating one.

To get your public URL, click Share from your collection’s flyout menu. Your collection’s public link is displayed under the Get Link tab.

Get Link - Postman

Using this link, you no longer need to export and move collection files. You do need to export the environment file when it is updated, but this happens less frequently.

Download your Postman environment by clicking the Manage Environments cog icon. Click Download to get the file. The filename will be something like User-API-Tests.postman_environment.json (depending on how you named the environment). Add this environment file to the root of your project.

Next, install the Newman CLI tool at the root of your project by running:

npm install --save-dev newman

Replace the test script in package.json with:

"scripts": {
    ...
    "test": "npx newman run [YOUR_COLLECTION_PUBLIC_URL] -e ./[YOUR_ENVIRONMENT_FILENAME].json"
}

This test script uses npx to run newman against the collection’s URL. The collection URL uses the environment’s filename to specify where the tests will run. Replace the values [YOUR_COLLECTION_PUBLIC_URL] and [YOUR_ENVIRONMENT_FILENAME] with your collection public URL and environment filename, respectively. Newman runs the tests defined in the collection against the requests and picks up any pre-request scripts that have been defined.

Your tests are set up for automation.

Connecting your project to CircleCI

Begin by pushing your project to GitHub.

Note: The cloned project may throw an error about having already been initialized as a git repo (or one that a remote repo is already contained). If this happens, run rm -rf .git to delete any existing git artifacts. Then re-initialize it with git init.

After you have committed all changes, go to the Add Projects page on the CircleCI dashboard to add the project.

Add Project - CircleCI

Click Set Up Project.

Add Config - CircleCI

On the setup page, click Use Existing Config to indicate that you are setting up a configuration file manually and not using the sample displayed. Next, you get a prompt to either download a configuration file for the pipeline, or to start building.

Build Prompt - CircleCI

Click Start Building. This build will fail because we have not set up our configuration file yet. We’ll do this in the next step.

Before you leave the CircleCI console, you need to set up environment variables to deploy the application to both environments (staging and production) on Heroku.

From the Pipelines page, select your application, then click Project Settings.

Project Settings - CircleCI

From the Project Settings side-menu, click Environment Variables, then clickAdd Environment Variable.

Add Environment variable - CircleCI

Add these variables:

  • HEROKU_STAGING_APP_NAME: The Heroku app name for the staging environment (in this case users-api-staging)
  • HEROKU_PRODUCTION_APP_NAME: The Heroku app name for the production environment (in this case users-api-production)
  • HEROKU_API_KEY: Your Heroku API key

Now you are ready to deploy to both the staging and production environments on Heroku.

Configuring a stage -> test -> approve -> deploy workflow

As promised at the beginning of this tutorial, our goal is to create an approval-based workflow in which an application is:

  1. Deployed to a staging environment
  2. Tested on the staging address and (if tests pass)
  3. The process waits for an admin to click an approval button in the CircleCI console and
  4. The deploy job is triggered and deploys the application to the production environment.

Our next task is to write a deployment pipeline script that performs those steps. Start by creating a folder named .circleci at the root of the project. Add a configuration file named config.yml inside the folder you just created. Enter the following code:

jobs:
  stage:
    executor: heroku/default
    steps:
      - checkout
      - heroku/install
      - heroku/deploy-via-git:
          app-name: $HEROKU_STAGING_APP_NAME

  test:
    working_directory: ~/repo
    docker:
      - image: circleci/node:10.16.3
    steps:
      - checkout
      - run:
          name: Update NPM
          command: "sudo npm install -g npm@5"
      - restore_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
      - run:
          name: Install Dependencies
          command: npm install
      - save_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
          paths:
            - ./node_modules
      - run:
          name: Run tests
          command: npm run test

  deploy:
    executor: heroku/default
    steps:
      - checkout
      - heroku/install
      - heroku/deploy-via-git:
          app-name: $HEROKU_PRODUCTION_APP_NAME
orbs:
  heroku: circleci/heroku@0.0.10
version: 2.1
workflows:
  stage_test_approve_deploy:
    jobs:
      - stage
      - test:
          requires:
            - stage
      - hold:
          type: approval
          requires:
            - test
      - deploy:
          requires:
            - hold

That is quite a script, right? Let me break it down and add some detail. We start by creating 3 jobs:

  • stage: Checks out the project code from the repo and uses the Heroku CircleCI orb to deploy the application to the staging environment.
  • test: Installs the project dependencies needed to have the newman CLI installed. It then runs the test script in package.json, invoking newman to run the tests in the collection against the staging application that was just deployed.
  • deploy: Deploys the application to the production environment on Heroku.

Next, the heroku orb is pulled in:

orbs:
  heroku: circleci/heroku@0.0.10

Orbs are reusable pipeline packages that abstract common tasks. The orb that we are using abstracts the process of installing and setting up the heroku CLI tool to deploy the application.

Finally, the script defines the stage_test_deploy workflow. This workflow runs the stage job. Once that job is run successfully, the test job runs tests against the application that was just deployed. If the test job is successful meaning all tests on the staging application passed, the workflow then runs a special hold job. The hold job pauses the workflow and waits for approval to be made before it continues. The hold job can be given any name you prefer. The important feature of this job is that its type is set to approval. This type indicates that it is an intermediary job, which means its main function is to pause the workflow until approval is complete.

An admin in the organization with the authority to approve clicks an approval button on the CircleCI console. That click causes the deploy job to continue, deploying the application to the production environment.

It is time to test our workflow. Commit all changes and push the code to the remote repository to run the pipeline script.

The currently active job is running while the others wait.

Workflow running - CircleCI

Once the test job is done, you will see the hold job pause the workflow for approval.

Workflow paused - CircleCI

Click the hold job to open the approval dialog.

Approval dialog - CircleCI

Click Approve to continue the workflow. The workflow will resume and the deploy job will run to push the application to production.

To review the steps in the process and their duration, just click the workflow (stage_test_deploy) link.

Workflow complete - CircleCI

You can also click each build for details about how it ran.

Test Job Details - CircleCI

Great work! Your tutorial project is approved.

Conclusion

In this tutorial we have shown how CircleCI makes it easy to implement approval structures in automated continuous integration processes. If approvals are part of your deployment processes, you can apply what you have learned here to other workflows, even to more complex scenarios.

Happy coding!


Fikayo is a fullstack developer and author with over a decade of experience developing web and mobile solutions. He is currently the Software Lead at Tech Specialist Consulting and develops courses for Packt and Udemy. He has a strong passion for teaching and hopes to become a full-time author.