As shortages of consumer goods and rising prices caused by bottlenecks in international supply networks have become more common, the global supply chain and its vulnerabilities have been top of mind for many. For developers, several high-profile software security exploits have recently underscored the risks inherent in a similar type of supplier network: the software supply chain.
Software applications are no longer built entirely from custom code. Instead, they are made up of a complex web of open source components and libraries, most of which inherit functionality from other third-party sources. This chain of dependencies empowers developers to make use of their preferred tools and enables teams to ship functional software to their users incredibly quickly, but it also exposes organizations and their customers to vulnerabilities introduced by changes outside of their direct control.
In this article, we will look closely at what the software supply chain is, what risks it presents to software producers, and how your organization can take advantage of continuous integration to automate security and compliance checks that can let you take full advantage of the open source ecosystem while mitigating supply chain risks.
with new triggers and permissions controls
What is the software supply chain?
It is hard to imagine any organization today that does not rely on multiple pieces of software for its day-to-day operations. In almost all cases, the software is built with varying dependencies on prebuilt external components. So, by default, the organization inherits the software supply chain of all parts of its software.
The software supply chain consists of code, configurations, proprietary and open source binaries, libraries, plugins, and container dependencies. It also includes building orchestrators and tools such as assemblers, compilers, code analyzers and repositories, security, monitoring, and logging ops tools. The software supply chain also encompasses the people, organizations, and processes involved in software development.
Because of how big and convoluted the software supply chain can get, risk-averse businesses and governments usually request a software bill of materials (SBOM) that describes a part or most of the supply chain. In May 2021, for example, the Biden administration issued an executive order requiring software vendors working with the federal government to provide an SBOM for their projects.
The insistence on SBOM is understandable given that in-house, outsourced, proprietary, or open source software often uses external components. While the apparent benefits to developing software this way — such as expedited development, lower production cost, and reduced time to market — are desirable, the threat of bad actors exploiting common vulnerabilities and exposures (CVEs) in those components is not.
Examples of software supply chain attacks
The large number of cyberattacks on software supply chains within the last two years illustrates the increased risk exposure that the software supply chain can introduce. At least four prominent attacks — SolarWinds, Codecov, Kaseya, and Log4j — occurred in 2021 alone.
In the SolarWinds attack between March and June 2020, an estimated 18,000 customers on the Orion platform, including a number of US government agencies, downloaded updates that were injected with malicious code. This code granted unauthorized backdoor access to systems and private networks. SolarWinds did not discover the breach until December 2020, prompting recalls, hotfixes, and an advisory to customers.
A few weeks later, in January 2021, an attacker obtained credentials used in Codecov’s Docker image creation process because of an error in the build process. The credentials allowed the modification of the bash uploader script. With the script, they stole credentials and additional resources in unsuspecting customers’ continuous integration (CI) environments. Codecov did not detect the breach until a few months later in April.
On July 2, 2021, some ninety days later, a sophisticated ransomware group exploited a vulnerability in Kaseya Virtual System Administrator (VSA) servers — affecting an estimated 1,500 small businesses. Unlike the cases mentioned above, Kaseya detected the breach the same day — perhaps because the attackers demanded ransoms from affected parties ranging from $45,000 to $5 million. Kaseya advised customers to shut down their VSA servers while they fixed the problem. Fortunately, patches were available as early as two days later, and were available until July 22, when Kaseya obtained a working decryption tool from a third party. Technical support continued, and an advisory was subsequently issued the following month.
In December of 2021, a few months after the Kaseya incident, what is arguably the most simple but pervasive software supply chain attack occurred. Following the disclosure of the proof of concept (POC) regarding remote code execution (RCE) in Apache Log4j, the popular Java logging framework, attackers began mass exploitation of a vulnerability. Named CVE-2021-44228, the vulnerability allowed attackers to push malware onto vulnerable Lightweight Directory Access Protocol (LDAP) servers. Although an update addressing the problem came relatively quickly, lingering log4j vulnerabilities have continued to pop up, prompting the recent Cybersecurity and Infrastructure Security Agency (CISA) advisory in April 2022.
Why is the software supply chain vulnerable to attack?
At its core, the software supply chain is an increasingly large, complex, and interconnected system of technology, people, and process touchpoints presenting multiple attack vectors. Bad actors can use these touchpoints to infiltrate the software supply chain.
The “technology” touchpoint generally consists of infrastructure, software, and codebases.
Infrastructure refers to virtual or physical equipment used to operate software. Infrastructure such as servers, virtual machines, storage, and networking devices are vulnerable to misconfiguration errors that leave critical resources exposed to cyberattacks. According to Aqua Security’s 2021 Cloud Security Report, 90 percent of organizations are at risk of security breaches due to misconfigured cloud infrastructure.
Software composed of proprietary code, but more so open source libraries and third-party tools, is vulnerable to inserting malicious code or exploiting bugs in code, orchestrating package dependency confusion, hijacking updates, and undermining code signing processes.
Codebases that house these programs can be vulnerable too. Synopsys reports as much as 88 percent of commercial code bases containing open source software have components that are behind on user updates by two years.
The “people” touchpoint consists of developers and other persons who either maliciously or inadvertently introduce vulnerabilities in the software supply chain. For example, the crossenv typosquatting incident in 2017 was a malicious attack on the npm registry to trick unsuspecting developers into installing a crossenv (instead of cross-env) package that contained malware.
“Processes” are touchpoints of interest to attackers, especially identity and access management (IAM) processes. Attackers can compromise IAM controls through employees or systems to introduce spyware and ransomware.
How to improve software supply chain security
The first step towards securing your software supply chain is to achieve visibility of the components. Vendors and end-users can do this with an SBOM that lists all third-party components and dependencies within the software you distribute and use.
An SBOM provides an overview of what is happening, demonstrates security awareness and license compliance, and can be a guide to the latest advisory affecting components in the software. You can obtain further visibility and security of your software supply chain by adopting various automated vulnerability scanning technologies.
It is not enough to scan for or track common vulnerabilities. How quickly and thoroughly you remediate vulnerabilities can make a difference in your level of exposure. As such, it helps to have a dedicated incident response team to provide prompt patches or updates as needed. Ensure that you have well-written failover processes that you periodically and rigorously test.
Only use trusted repositories and verified sources for suppliers in the chain and perform regular risk assessments of libraries, frameworks, and suppliers. Be sure to complement the supplier’s testing with your independent testing frequently. As a vendor, you can implement strong IAM policies and controls with the principle of least privilege in mind, including data governance guidelines to protect your data and infrastructure within your software supply chain.
How to automate supply chain security with CI/CD
The best way to automate the security of your supply chain is with a robust continuous integration and continuous delivery (CI/CD) pipeline. With CircleCI, you can seamlessly integrate application security (AppSec) and development security operations (DevSecOps) tools to check for vulnerabilities from the version control system (VCS) through the build, test, and deployment stages of your pipeline.
At the VCS stage, developers might make the mistake of committing secrets in plain text to a repository, which an attacker can uncover in Git history. CircleCI integrates with your preferred VCS provider and enables you to scan commits for secrets using relevant orbs such as the GitGuardian orb. You can securely store and manage your secrets within CircleCI with encrypted-at-rest environment variables or with container-stored contexts for use across projects.
Alternatively, you can fetch your stored secrets from third-party solutions dynamically or commit the encrypted version and keep the decryption tool in a context so that you can execute a decryption job to retrieve the secrets at the build and test stages in your CI/CD pipeline.
At the VCS stage, you can also use the Lightspin orb to scan and provide advice for remediation on IAM misconfigurations, exposed credentials, and insecure configurations in infrastructure as code (IaC) templates found in your codebase.
At the build stage of your pipeline, you can run static application security testing (SAST) jobs on your code and open source libraries using relevant orbs. For example, you can use the Snyk orb to scan your codebase for dependency vulnerabilities. If the scan reveals a potential threat in your software supply chain, your build will fail and Snyk will output recommendations for improving the security of your code.
At the deployment stage, you can run dynamic application security testing (DAST) jobs to catch vulnerabilities at runtime in production. For example, the DeepFactor orb provides prioritized insights on application code, package dependencies, web APIs, and compliance CVEs based on app behavior.
Conclusion
Software supply chain attacks are increasingly rampant. And with the interconnectedness of modern businesses, coupled with the increasing rate of change in the software ecosystem, bad actors have plenty of attack points to target. This underscores the need to prevent successful vulnerability exploitation as much as possible.
One way to do this is to add automated supply chain security scans at every point of your software delivery process with CI/CD.
Using CircleCI, you can create workflows with jobs that perform vulnerability scans and provide advisory on codebases, open source libraries, dependencies, and other third-party tools. You can detect IaC misconfigurations in infrastructure in infrastructure, securely build and deploy artifacts, and validate compliance stipulations in your CI/CD pipeline using a variety of automatic pipeline triggers.
To stay ahead of attackers and learn more about adding automated supply chain security to your CI/CD pipeline, get started with a free CircleCI account today.
