HashiCorp company logo in white

HashiCorp cuts test suite time by 75% with CircleCI

Cloud infrastructure provider, HashiCorp, adopts CircleCI to gain extensive build insights and increased security

The challenge

HashiCorp software helps companies transition from on-premise infrastructure to operating in the cloud, with help from its suite of multi-cloud infrastructure automation products. The company’s 800 developers have complex engineering tasks to solve in the course of updating code for the platform and products – such as testing code in a client-server environment and the need for a large number of CPUs.

The complexity of the HashiCorp development environment demanded more time and management from engineers, says Alvin Huang, Release Engineering Lead for the business.

“Some of our CI jobs have to run on very large resource classes,” Huang explains. “There are some build jobs that require a large number of CPUs that can’t be parallelized across workers, such as UI builds.” In addition, many HashiCorp products run in a client-server model and we needed a CI system that would facilitate setting up these instances and networking.

Huang and his colleagues closely watched metrics for releasing new code as they used TeamCity, TravisCI, and GitHub Actions.

“The most important metric is being able to publish releases on the day they’re scheduled with the least friction,” Huang says. “It was also important that the feedback loop for developers is as tight as possible.”

The solution

The problems associated with too many configurations and too many CPUs accelerated the search for a new CI/CD solution.

“Rather than running builds across multiple CI providers and having to manage multiple configuration files, we wanted to consolidate on one platform that gave us the ability to scale across multiple runners and build out reusable components like orbs. We thought the better security in CircleCI would be a huge win for us.”

Another benefit of CircleCI is the ability to SSH into build containers. “That makes it easier for engineers to debug when tests are failing, or when artifacts don’t end up in the right place,” Huang says. “Other CI/CD providers wouldn’t give us this capability.”

The CircleCI Insights dashboard is already helping Huang and his engineering colleagues keep track of test results as a measure of build success.

“The dashboard summarizes what our historical test results look like – which tests are the flakiest and which ones are the slowest,” he says. “Then we can tackle those specific tests instead of just guessing.”


Alvin Huang
“Rather than running builds across multiple CI providers and having to manage multiple configuration files, we wanted to consolidate on one platform that gave us the ability to scale across multiple runners and build out reusable components like orbs. We thought the better security in CircleCI would be a huge win for us.”

Alvin Huang
Release Engineering Lead at HashiCorp


How CircleCI powers HashiCorp’s business

Since the release engineering team began using CircleCI, test suites that used to take as long as 45 minutes now take just 10 to 15 minutes, due to parallelism and test splitting.

“We can now build multiple binaries concurrently rather than serially,” Huang says. “We use parallelism across all of our builds of various OS/ARCH combinations, as well as test-splitting across workers for large test suites. This enables a faster feedback loop for our developers.”

The feedback has also become valuable as engineers strengthen their knowledge of what’s needed for fast, high-quality builds. “We can leverage the context and learnings of builds over time,” Huang says. “We no longer have to start from scratch when we build out configuration files. We can leverage orbs and standard practices to bootstrap new projects quickly.”

Benefits since using CircleCI

  • 75% reduction in test suite times
  • Improved security
  • Greater knowledge of builds using Insights dashboard

About HashiCorp

HashiCorp is a leader in multi-cloud infrastructure automation software. The HashiCorp software suite enables organizations to adopt consistent workflows and create a system of record for automating the cloud: infrastructure provisioning, security, networking, and application deployment.

close

Thank You for Submitting Your Info


You should receive an automated response notifying you that we received your info. Someone from our Enterprise team will be reaching out to you shortly.


CircleCI Success Logo