Security is a vital part of application development, yet it may be neglected until an attacker takes advantage of a vulnerability in the system. The consequences of a security breach can damage an application’s integrity as well as a company’s reputation and revenue. Software architects and engineers need to pay special attention to securing the systems they work on.

Running security-based tests and automating security into the CI/CD workflow is the inspiration behind DevSecOps, an extension of the DevOps methodology into the field of application security.

In this article, I will lead you through automating security checks in an application. I will show you how to run a test on a web form that injects the URL field with malicious domains. The goal of the test is to break the web form. To stop the attack, the web form has been protected with logic that checks to see if a user is trying to enter a malicious domain. I will show you how to use scheduled pipelines to run the security tests on a regular schedule.

Prerequisites

To follow along with this tutorial, you will need these in place:

Once you have these set up, you can begin.

Getting Webshrinker credentials

Webshrinker is an AI-powered domain categorization system owned by DNS security company DNSFilter. Webshrinker is capable of identifying threat domains and labeling them according to their threat categories. Phishing, malware, and botnet are just three of the threat types Webshrinker can identify.

The form we are protecting in the project takes a fully qualified domain name in its URL field. It sends the domain name to the Webshrinker API to be scanned for threats. If the results indicate that it is malicious, Webshrinker returns a threat identifier. The form uses the threat identifier to deny the processing of the domain entry.

You will need an API key and API secret to use Webshrinker in this demo. Once you have an account created, you can create new API credentials by going to the API Access Keys page and clicking the Create API Key button. That generates your API secret and token.

You will use your credentials in the next step.

Cloning and running the demo application

The first step is to run the demo web form locally and inspect its behavior. You can clone the code for the form from this repository by running this command from anywhere on your system:

git clone -b base-project --single-branch https://github.com/coderonfleek/scheduled-security-scan.git

Once you have the code on your system, install the dependencies by running this command:

cd scheduled-security-scan
npm install

Now go to lines 65 and 66 in the index.html file at the root of the project and replace the placeholders with your API key and API secret, respectively.

When the installation of dependencies is complete, run the application with this command:

npm start

The application boots up, and you can view the web form at https://localhost:5000.

Enter an email address into the email field. In the URL field, enter a safe domain like facebook.com and click Submit. You will get a safe response on the right side of the form.

Regular domain test

Now, test the URL field of the form using this threat-based domain name: selcdn.ru. Please, do not enter this domain name directly in your browser. You will get a threat alert message.

Malicious Domain test

Adding the security tests

The next step is to write tests that perform the security checks automatically. You will be adding some automated functional tests using Google’s Puppeteer. Puppeteer simulates the way a real-world user would fill the form.

Add a new file named login.test.js at the root of the project and enter this code:

const puppeteer = require("puppeteer");

const user_email = "test@example.com";
const non_threat_site = "facebook.com";
const malicious_site = "selcdn.ru";
const phishing_site = "mail.glesys.se";
const expected_safe_site_message = "Entry clean, process form";
const expected_threat_site_message = "Threat Detected. Do not Process";

test("Check Non-threat Site Entry", async () => {
  const browser = await puppeteer.launch();
  try {
    const page = await browser.newPage();

    await page.goto("http://localhost:5000");

    await page.type("#userEmail", user_email);
    await page.type("#userSite", non_threat_site);
    await page.click("#submitButton");

    let messageContainer = await page.$("#infoDisplay");
    await page.waitForTimeout(4000);
    let value = await messageContainer.evaluate((el) => el.textContent);

    console.log(value);

    expect(value).toBe(expected_safe_site_message);
  } finally {
    await browser.close();
  }
}, 120000);

test("Check Malicious Site Entry", async () => {
  const browser = await puppeteer.launch();
  try {
    const page = await browser.newPage();

    await page.goto("http://localhost:5000");

    await page.type("#userEmail", user_email);
    await page.type("#userSite", malicious_site);
    await page.click("#submitButton");

    let messageContainer = await page.$("#infoDisplay");
    await page.waitForTimeout(4000);

    let value = await messageContainer.evaluate((el) => el.textContent);

    console.log(value);

    expect(value).toBe(expected_threat_site_message);
  } finally {
    await browser.close();
  }
}, 120000);

This file contains two test cases. The first checks that non-threat domains like facebook.com are not blocked by the system. This keeps your security implementation from overzealously blocking regular domains.

The second test case checks for malicious entries by using a sample malicious domain. If the form blocks this domain, the test passes. If the malicious domain is not blocked, the test fails.

Save the file and go to the root of the project in your terminal. Make sure that the application is running in another shell; the tests need it to run successfully. Run this command:

npm run test

Once the test run is complete, you will have results in your CLI like these:

MACs-MBP-2:scheduled-security-scan mac$ npm run test

> xss-attack@1.0.0 test /Users/mac/Documents/CircleCiProjects/2022/scheduled-security-scan
> jest

  console.log
    Entry clean, process form

      at Object.<anonymous> (login.test.js:26:13)

 PASS  ./login.test.js (109.034 s)
  ✓ Check Non-threat Site Entry (80318 ms)
  ✓ Check Malicious Site Entry (8210 ms)

Test Suites: 1 passed, 1 total
Tests:       2 passed, 2 total
Snapshots:   0 total
Time:        120.252 s
Ran all test suites.
  console.log
    Threat Detected. Do not Process

      at Object.<anonymous> (login.test.js:54:13)

Developers can be tempted to write tests that match their code capabilities. As a best practice, be sure to separate the development and testing teams. This practice lets the testing team write exhaustive tests and protects developers from themselves.

Adding the pipeline configuration script

To automate the testing process, we can build a continuous integration (CI) pipeline using CircleCI.

To set up the CI pipeline, you need a pipeline configuration script. You can add this script into a .circleci/config.yml file. Create the file at the root of the project and enter:

version: 2.1
jobs:
  build:
    working_directory: ~/repo
    docker:
      - image: cimg/node:14.18-browsers
    steps:
      - checkout
      - run:
          name: Update NPM
          command: "sudo npm install -g npm"
      - restore_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
      - run:
          name: Install Dependencies
          command: npm install
      - save_cache:
          key: dependency-cache-{{ checksum "package-lock.json" }}
          paths:
            - ./node_modules
      - run:
          name: Run the application
          command: node server.js
          background: true
      - run:
          name: Run tests
          command: npm run test

This configuration script:

  • Pulls in a Docker image with browsers and Node.js installed
  • Updates npm and installs the required project dependencies
  • Runs the application in the background so that the Cypress tests can load it
  • Runs the tests

Save the file and push the project to GitHub

Next, add the repository as a CircleCI project.

Add Project - CircleCI

Select Branch - CircleCI

Once the project is set up on CircleCI, the tests run immediately. When the build is done, you will get a successful build status.

Build Success - CircleCI

Click the Build link to review the build details.

Build Details - CircleCI

Setting up a scheduled pipeline on CircleCI

Usually, your CI pipeline runs only when a new commit is made to the remote code repository you used when you set up the pipeline. To get the most from security scans like the tests we have written, the security state of the application should be updated even when no new code has been pushed.

That means our pipeline needs to run at regular intervals, like a cron job, to accurately report the security status of the application. To keep things efficient, it is better not to mix security tests with functionality tests that check for bugs or validate application functions.

With scheduled pipelines on CircleCI, you can configure your pipelines to run on just one specific day or on all days of the week at specific times.

For this tutorial, we will set up our pipeline to run every five minutes every day of the week. The interval set for this project is not based on any real-life situation, detailed consideration, or best practice. It is just to demonstrate the pipeline running periodically during the demo period.

To configure your pipeline to run on a schedule, go to CircleCI and select Project Settings, then Triggers.

On the Triggers page, click Add Scheduled Trigger to display the trigger form. Fill in the form to configure the pipeline to run every five minutes, every day.

Set up trigger - CircleCI

Click Save Trigger and your trigger will be created.

Return to the Pipelines page and wait for at least five minutes. Your pipeline will be triggered by the CircleCI system.

Scheduled Runs

The five-minute intervals between each run indicate that the configuration is in full effect.

Conclusion

In this tutorial, I have demonstrated how to run periodic security scans on your application using CircleCI’s scheduled pipelines. Instead of waiting for new code to build, you can have the system run the security checks constantly and report back when something is broken or vulnerable to a breach.

Security scans are just one of the many uses cases for this kind of automation. Clean-up tasks like log and data archiving are also great candidates for configuring a schedule. Share what you have learned with your team and start experimenting with scheduled pipelines.

Happy coding!


Fikayo Adepoju is a LinkedIn Learning (Lynda.com) Author, Full-stack developer, technical writer, and tech content creator proficient in Web and Mobile technologies and DevOps with over 10 years experience developing scalable distributed applications. With over 40 articles written for CircleCI, Twilio, Auth0, and The New Stack blogs, and also on his personal Medium page, he loves to share his knowledge to as many developers as would benefit from it. You can also check out his video courses on Udemy.