This article was originally posted on Forestry.io’s blog. Forestry.io is a Git-backed CMS (content management system) providing editors and other non-technical users with a custom UI for websites and web products built using static site generators.
Tools like Hugo, Jekyll, and Gatsby have made building static sites a popular and practical choice for developers. One major disadvantage these tools have, however, is the need to regenerate and redeploy their files every time there is new content to publish.
Automating this process will go a long way toward making your static site feel like a dynamic CMS. It will also save you time and improve the reliability of your deployments, as the exact same steps will run every time you deploy. For this reason, automated deployment is a cornerstone of modern web development.
Our favorite deployment tool is CircleCI, we’re using it at Forestry.io every day to deploy our Hugo site. For our tutorial today we’ll be using CircleCI to deploy a Hugo site but you can use CircleCI for any static site that needs automated deployment.
Continuous Integration Vs Continuous Deployment
Continuous Integration (CI) and Continuous Deployment (CD) describe practices for automating the release of software. The two terms are semantically different, but go hand in hand. You will often see them referenced together or used interchangeably.
Continuous Integration minimizes integration problems by constantly integrating and testing changes to the code. This is achieved by running automated tests whenever code is contributed to a project.
Continuous Deployment is the practice of automatically releasing updates to production. CD is an extension of CI — you will want to run your integration tests before deploying your code, relying on automated testing to ensure the integrity of the production software.
If you’ve never attempted to set up an automatic deployment before, the task may seem daunting. Fortunately, there are many CI/CD services available that make this easy. In this post, I will show you how to use CircleCI to automatically deploy a Hugo site to an Amazon S3 bucket.
Build, Test, Deploy
Most automated deployments can be broken down into three basic steps. First, one or more build steps will be run. These steps will perform tasks like installing external dependencies, compiling or preprocessing source code, and minifying or optimizing assets. Once the build is complete, the code should be ready to ship. At this point, tests may be run to check for errors. If the test results are acceptable, the code is safe to deploy. Deploying the code sends your project files from the CI environment to the destination server.
Using CircleCI
To start using CircleCI, sign up and link your GitHub account. This will enable you to hook up any of your GitHub projects to CircleCI. Click on Projects in the left-hand navigation and then click on the Add Project button. You will see a list of your GitHub projects. From there, select the Hugo project you want to create a deployment for. Choose Linux for your operating system, and 2.0 for the platform. Then, scroll down and click the Start Building button. Your first build will fail because we haven’t added a configuration for it, but your project build history will now be available under Builds in the left-hand navigation. Now that CircleCI knows about your project, it’s time to configure the deployment.
Configuring Your Deployment
Create a new file in your repository at .circleci/config.yml
. CircleCI will use this file for its deployment instructions. Copy the following YAML into this file to use as a starting point:
version: 2
jobs:
build:
docker:
working_directory:
environment:
steps:
In the version
section we tell CircleCI that we are using version 2 of their configuration API. CircleCI 2.0 enables us to define a Docker image to build our project in, giving us a lot more control over our execution environment.
Under jobs
we specify the jobs that we want to run when code is pushed. You can configure multiple jobs in your config and orchestrate them using the workflows feature, but we won’t be doing that in this tutorial. Since the deployment process for our static site is relatively basic and we must run all of our jobs sequentially (build, then test, then deploy,) we can keep things simple by using a single job. Because we are only using a single job, CircleCI requires that we name it build
.
Our working_directory
is the location within the container that our build commands will be run. We will set this to ~/hugo
.
We can use the environment
section to define environment variables that will be available to our build commands. In this section we will define a variable HUGO_BUILD_DIR
and use that to reference the destination directory where Hugo should place our generated HTML. Let’s set this to ~/hugo/public
.
version: 2
jobs:
build:
docker:
working_directory: ~/hugo
environment:
HUGO_BUILD_DIR: ~/hugo/public
steps:
The Docker Image
docker:
- image: cibuilds/hugo:latest
CircleCI uses Docker to provide disposable environments to build and test your code in. We can define one or more Docker images to use for our build in the docker
section. Since we don’t require any additional services to run our deployment, we will only specify one Docker image. Check out the CircleCI documentation if you’re curious about how and when to use multiple Docker container images. If Docker is completely new to you, you might want to take a look at Docker’s explanation of containers to understand what’s going on here.
We are going to use the cibuilds/hugo
image as the base for our Docker container. It has Hugo pre-installed, which we will need in order to build our Hugo app. We will need a few other dependencies, but we can install them during our build process.
version: 2
jobs:
build:
docker:
- image: cibuilds/hugo:latest
working_directory: ~/hugo
environment:
HUGO_BUILD_DIR: ~/hugo/public
steps:
Configuring Your Build Steps
The steps
section is where we add a list of commands needed to build, test, and deploy our project.
1 - Install Git and checkout the repository
- run: apk update && apk add git
- checkout
run
and checkout
are CircleCI commands — they are the interface through which we can send instructions to the execution environment. We will be using run
heavily: it allows us to specify commands to be run in the execution environment. checkout
is a special step that CircleCI provides to simplify checking out the project into your execution environment’s working_directory
.
Our cibuilds/hugo
image is built on Alpine Linux, so we use the apk
command to interact with the OS’s package manager. We run apk update
to update the package index with the latest available packages, and then apk add git
to install Git. Once Git is installed, we can run the checkout
step.
2 - Install Submodules
- run: git submodule sync && git submodule update --init
If you’re using Git submodules to manage any third party dependencies, you will need to run this step to install them.
3 - Install awscli
- run: apk add --update python python-dev py-pip build-base
- run: pip install awscli
These commands install the awscli
utility, which we will use to deploy the files to S3. We must first install pip
, Python’s package manager, to install awscli
.
4 - Build With Hugo
- run: HUGO_ENV=production hugo -v -d $HUGO_BUILD_DIR
At this point, we have all of our source code in our execution environment. It’s time to build! We tell Hugo to generate the files in $HUGO_BUILD_DIR
, which is the environment variable we declared earlier in our config.
5 - Test With Htmlproofer
- run: |
htmlproofer $HUGO_BUILD_DIR --allow-hash-href --check-html \
--empty-alt-ignore --disable-external
This Docker image comes with html-proofer already installed, so we just have to run the htmlproofer
command in our $HUGO_BUILD_DIR
to test our generated HTML files.
6 - Prepare For Deployment
After passing our tests, the code is now ready to deploy. Before we run the deploy command, however, we need to make sure CircleCI can communicate with our deployment target.
Connect to S3
In order to deploy to S3, we will need to create an IAM user that can write to our bucket. The following security policy will allow CircleCI to send files to the bucket:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::your-bucket-name",
"arn:aws:s3:::your-bucket-name/*"
]
}
]
}
Make note of the Access Key ID and Secret Access Key of this user. To provide this information to CircleCI, access your project’s build settings by clicking the cog next to the project name on the Builds screen. On the settings screen, locate the AWS Permissions link under the Permissions section. Here we can add the credentials for our IAM user.
7 - Deploy to Production Environment
We will use CircleCI’s deploy
command to ship the code. deploy
works just like the run
command, but should be used instead of run
for deploying code. CircleCI can be configured to run some steps in parallel, but any deploy
steps will wait for parallel execution to finish and ensure that all tasks have completed successfully before running. Because of this, deploy
should always be used when it’s time to move code out of the execution environment.
- deploy:
name: deploy to AWS
command: |
if [ "${CIRCLE_BRANCH}" = "master" ]; then
aws s3 sync $HUGO_BUILD_DIR \
s3://your-bucket-name/your-subfolder --delete
else
echo "Not master branch, dry run only"
fi
Things are pretty straightforward from here. This is a multi-line command so we have to use the |
character to indicate this. We’re using a conditional to check the CIRCLE_BRANCH
environment variable and only deploy if we’re building the master branch. Using this strategy, our configuration will continue to build and test code pushed to other branches, but will not deploy them to our production environment.
Deploy Continuously
At this point, we’re all done! Your project should now build, test, and deploy successfully whenever code is pushed to the GitHub repo.
The Final Config File
version: 2
jobs:
build:
docker:
- image: cibuilds/hugo:latest
working_directory: ~/hugo
environment:
HUGO_BUILD_DIR: ~/hugo/public
steps:
# install git
- run: apk update && apk add git
# checkout the repository
- checkout
# install git submodules for managing third-party dependencies
- run: git submodule sync && git submodule update --init
- run:
name: install AWS CLI (first install pip, the Python package manager)
command: |
apk add --update python python-dev py-pip build-base
pip install awscli
# build with Hugo
- run: HUGO_ENV=production hugo -v -d $HUGO_BUILD_DIR
- run:
name: test our generated HTML files
command: |
htmlproofer $HUGO_BUILD_DIR --allow-hash-href --check-html \
--empty-alt-ignore --disable-external
# `deploy` step: identical to a `run` step, but uses only one container:
# https://circleci.com/docs/configuration-reference/#deploy
- deploy:
name: deploy to AWS
command: |
if [ "${CIRCLE_BRANCH}" = "master" ]; then
aws s3 sync $HUGO_BUILD_DIR \
s3://your-bucket-name/your-subfolder --delete
else
echo "Not master branch, dry run only"
fi
If things aren’t working quite right for you, don’t panic! Automating something is always more difficult than doing it manually. It’s worth taking the time up front to figure it out, as it will save you lots of work and frustration in the long run. Take a look at CircleCI’s documentation and community forum if you need help.
For static sites, automated deployment is essential to providing a seamless publishing experience. Services like CircleCI have made automation accessible to developers at any level. It’s time to introduce automation into your static site workflow — once you do, you’ll wonder how you ever got along without it.