Security is a vital part of application development, yet it may be neglected until an attacker takes advantage of a vulnerability in the system. The consequences of a security breach can damage an application’s integrity as well as a company’s reputation and revenue. Software architects and engineers need to pay special attention to securing the systems they work on.
Running security-based tests and automating security into the CI/CD workflow is the inspiration behind DevSecOps, an extension of the DevOps methodology into the field of application security.
In this article, I will lead you through automating security checks in an application. I will show you how to run a test on a web form that injects the URL field with malicious domains. The goal of the test is to break the web form. To stop the attack, the web form has been protected with logic that checks to see if a user is trying to enter a malicious domain. I will show you how to use scheduled pipelines to run the security tests on a regular schedule.
Prerequisites
To follow along with this tutorial, you will need these in place:
- Node.js installed on your system (version 12 or greater)
- Git installed on your system
- A Webshrinker account; sign up for a free trial here
- A CircleCI account
Once you have these set up, you can begin.
Getting Webshrinker credentials
Webshrinker is an AI-powered domain categorization system owned by DNS security company DNSFilter. Webshrinker is capable of identifying threat domains and labeling them according to their threat categories. Phishing, malware, and botnet are just three of the threat types Webshrinker can identify.
The form we are protecting in the project takes a fully qualified domain name in its URL field. It sends the domain name to the Webshrinker API to be scanned for threats. If the results indicate that it is malicious, Webshrinker returns a threat identifier. The form uses the threat identifier to deny the processing of the domain entry.
You will need an API key and API secret to use Webshrinker in this demo. Once you have an account created, you can create new API credentials by going to the API Access Keys page and clicking the Create API Key button. That generates your API secret and token.
You will use your credentials in the next step.
Cloning and running the demo application
The first step is to run the demo web form locally and inspect its behavior. You can clone the code for the form from this repository by running this command from anywhere on your system:
git clone -b base-project --single-branch https://github.com/coderonfleek/scheduled-security-scan.git
Once you have the code on your system, install the dependencies by running this command:
cd scheduled-security-scan
npm install
Now go to lines 65 and 66 in the index.html file at the root of the project and replace the placeholders with your API key and API secret, respectively.
When the installation of dependencies is complete, run the application with this command:
npm start
The application boots up, and you can view the web form at https://localhost:5000.
Enter an email address into the email field. In the URL field, enter a safe domain like facebook.com and click Submit. You will get a safe response on the right side of the form.
Now, test the URL field of the form using this threat-based domain name: selcdn.ru. Please, do not enter this domain name directly in your browser. You will get a threat alert message.
Adding the security tests
The next step is to write tests that perform the security checks automatically. You will be adding some automated functional tests using Google’s Puppeteer. Puppeteer simulates the way a real-world user would fill the form.
Add a new file named login.test.js at the root of the project and enter this code:
const puppeteer = require("puppeteer");
const user_email = "test@example.com";
const non_threat_site = "facebook.com";
const malicious_site = "selcdn.ru";
const phishing_site = "mail.glesys.se";
const expected_safe_site_message = "Entry clean, process form";
const expected_threat_site_message = "Threat Detected. Do not Process";
test("Check Non-threat Site Entry", async () => {
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
await page.goto("http://localhost:5000");
await page.type("#userEmail", user_email);
await page.type("#userSite", non_threat_site);
await page.click("#submitButton");
let messageContainer = await page.$("#infoDisplay");
await page.waitForTimeout(4000);
let value = await messageContainer.evaluate((el) => el.textContent);
console.log(value);
expect(value).toBe(expected_safe_site_message);
} finally {
await browser.close();
}
}, 120000);
test("Check Malicious Site Entry", async () => {
const browser = await puppeteer.launch();
try {
const page = await browser.newPage();
await page.goto("http://localhost:5000");
await page.type("#userEmail", user_email);
await page.type("#userSite", malicious_site);
await page.click("#submitButton");
let messageContainer = await page.$("#infoDisplay");
await page.waitForTimeout(4000);
let value = await messageContainer.evaluate((el) => el.textContent);
console.log(value);
expect(value).toBe(expected_threat_site_message);
} finally {
await browser.close();
}
}, 120000);
This file contains two test cases. The first checks that non-threat domains like facebook.com are not blocked by the system. This keeps your security implementation from overzealously blocking regular domains.
The second test case checks for malicious entries by using a sample malicious domain. If the form blocks this domain, the test passes. If the malicious domain is not blocked, the test fails.
Save the file and go to the root of the project in your terminal. Make sure that the application is running in another shell; the tests need it to run successfully. Run this command:
npm run test
Once the test run is complete, you will have results in your CLI like these:
MACs-MBP-2:scheduled-security-scan mac$ npm run test
> xss-attack@1.0.0 test /Users/mac/Documents/CircleCiProjects/2022/scheduled-security-scan
> jest
console.log
Entry clean, process form
at Object.<anonymous> (login.test.js:26:13)
PASS ./login.test.js (109.034 s)
✓ Check Non-threat Site Entry (80318 ms)
✓ Check Malicious Site Entry (8210 ms)
Test Suites: 1 passed, 1 total
Tests: 2 passed, 2 total
Snapshots: 0 total
Time: 120.252 s
Ran all test suites.
console.log
Threat Detected. Do not Process
at Object.<anonymous> (login.test.js:54:13)
Developers can be tempted to write tests that match their code capabilities. As a best practice, be sure to separate the development and testing teams. This practice lets the testing team write exhaustive tests and protects developers from themselves.
Adding the pipeline configuration script
To automate the testing process, we can build a continuous integration (CI) pipeline using CircleCI.
To set up the CI pipeline, you need a pipeline configuration script. You can add this script into a .circleci/config.yml file. Create the file at the root of the project and enter:
version: 2.1
jobs:
build:
working_directory: ~/repo
docker:
- image: cimg/node:14.18-browsers
steps:
- checkout
- run:
name: Update NPM
command: "sudo npm install -g npm"
- restore_cache:
key: dependency-cache-{{ checksum "package-lock.json" }}
- run:
name: Install Dependencies
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package-lock.json" }}
paths:
- ./node_modules
- run:
name: Run the application
command: node server.js
background: true
- run:
name: Run tests
command: npm run test
This configuration script:
- Pulls in a Docker image with browsers and Node.js installed
- Updates
npmand installs the required project dependencies - Runs the application in the background so that the Cypress tests can load it
- Runs the tests
Save the file and push the project to GitHub
Next, add the repository as a CircleCI project.
Once the project is set up on CircleCI, the tests run immediately. When the build is done, you will get a successful build status.
Click the Build link to review the build details.
Setting up a scheduled pipeline on CircleCI
Usually, your CI pipeline runs only when a new commit is made to the remote code repository you used when you set up the pipeline. To get the most from security scans like the tests we have written, the security state of the application should be updated even when no new code has been pushed.
That means our pipeline needs to run at regular intervals, like a cron job, to accurately report the security status of the application. To keep things efficient, it is better not to mix security tests with functionality tests that check for bugs or validate application functions.
With scheduled pipelines on CircleCI, you can configure your pipelines to run on just one specific day or on all days of the week at specific times.
For this tutorial, we will set up our pipeline to run every five minutes every day of the week. The interval set for this project is not based on any real-life situation, detailed consideration, or best practice. It is just to demonstrate the pipeline running periodically during the demo period.
To configure your pipeline to run on a schedule, go to CircleCI and select Project Settings, then Triggers.
On the Triggers page, click Add Scheduled Trigger to display the trigger form. Fill in the form to configure the pipeline to run every five minutes, every day.
Click Save Trigger and your trigger will be created.
Return to the Pipelines page and wait for at least five minutes. Your pipeline will be triggered by the CircleCI system.
The five-minute intervals between each run indicate that the configuration is in full effect.
Conclusion
In this tutorial, I have demonstrated how to run periodic security scans on your application using CircleCI’s scheduled pipelines. Instead of waiting for new code to build, you can have the system run the security checks constantly and report back when something is broken or vulnerable to a breach.
Security scans are just one of the many uses cases for this kind of automation. Clean-up tasks like log and data archiving are also great candidates for configuring a schedule. Share what you have learned with your team and start experimenting with scheduled pipelines.
Happy coding!
Fikayo Adepoju は、Web とモバイル テクノロジー、DevOps に精通した LinkedIn Learning (Lynda.com) 講師、フルスタック開発者、テクニカル ライター、テクニカル コンテンツ クリエイターです。スケーラブルな分散アプリケーション開発については 10 年以上の経験を持っています。 CircleCI、Twilio、Auth0、The New Stack のブログで 40 以上の記事を執筆するほか、個人の Medium ページでも情報を発信しており、役立つ知識を多くの開発者に広めることに専心しています。 また、Udemy で動画形式のコース (英語) も開講しています。ぜひご覧ください。
