The ultimate goal of building and testing software is generally to deploy that software. The official CircleCI blog and many community blogs showcase examples on deploying to Kubernetes clusters, pushing to Docker registries, uploading to AWS S3, and even new serverless techniques. However, for solo developers or even small companies, using large deployment platforms like these is not always practical.
But there are other options.
In this blog post we’re going to cover a few ways to deploy your software to traditional Virtual Private Servers (VPS) such as those provided by Linode, DigitalOcean, or Vultr.
All of the tools we’re going to cover will send your data over the internet in order to deploy your software. That being the case, having a secure connection is very important. Many examples in this post will use SSH keys to encrypt the connection. I suggest creating a brand new SSH keypair specifically for CircleCI. My favorite resource on creating SSH keys is by GitHub.
The private key should be stored on CircleCI via the web app.
We have instructions you can follow here.
The public key should be stored in the
authorized_keys file for the user on the remote server you’d like to deploy to.
For example, if you were deploying to a Linux server
email@example.com, the public key should be appended to the file
/home/prod/.ssh/authorized_keys on that remote machine.
rsync is a file transfer tool that has been around for a while.
The name comes from “remote synchronization,” which means it allows you to transfer files from one directory or machine to another, but also keep them in sync.
This is useful because this purpose means that
rsync can compare files on a remote server and not transfer for them if they already exist, making for faster and more efficient deployments.
This works very well when you have several files to transfer, some updated and some not, such as when building a statically-generated website with Hugo or Jekyll. Basic Rails, Flask, and React apps can be deployed this way as well.
rsync -va --delete ./my-files/ firstname.lastname@example.org:/var/www/html
rsync, you provide options, the file or directory you want to transfer, and then where you want to transfer to.
-va- The ‘v’ means turn on verbosity so that we can see what’s happening in case something goes wrong, while the ‘a’ means archive mode. Archive mode turns on many useful options such as recursion, the copying of file permissions and ownsership, and more.
--delete- This tells
rsyncthat if a file has been deleted on CircleCI, it should be deleted from the remote server as well. Old files hanging around can lead to unsual issues where production doesn’t match your local environment, causing unwanted effects.
./my-files/- The directory whose contents we want to copy. Notice that there’s a trailing slash, though with
rsync, no trailing slash.
./my-fileswithout the trailing slash means “copy this whole directory including everything in it.”
./my-files/with the trailing slash means “only copy the contents of the directory and not the directory itself.”
email@example.com- ‘user’ is the username on the remote server ‘example.com’ that we want to deploy to.
:/var/www/html- This is the file path on the remote server where
rsyncshould be sending the files to. For web servers, this should be the DocumentRoot.
While some Docker images may already have
rsync installed, the official CircleCI Images and the third-party CI Builds Images don’t.
However, installing it is fairly straightforward:
# Ubuntu & Debian based Docker images sudo apt-get update && sudo apt-get install rsync # Alpine based Docker images apk update && apk add rsync
Here’s a snippet of the CircleCI config for my personal website (Feliciano.Tech), which is deployed via
deploy: docker: - image: cibuilds/hugo:0.42.1 steps: - attach_workspace: at: src - add_ssh_keys - run: | echo 'web01.revidian.net ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJiGRY6N9WYQ0vy6cTiwAgNbc6ueJmVo/EafBtmT7bcD6cQMbipYM/KfYQ2lCn2TxqWepZKYoyoVQXgArycCOns=' >> ~/.ssh/known_hosts rsync -va --delete src/public/ firstname.lastname@example.org:www/feliciano.tech/public_html
There are other options aside from
rsync which work similarly.
scp, known as Secure Copy, and
sftp, known as Secure FTP, are two tools that both utilize SSH for encryption and handle the basic file transfer process that you can use to deploy to a VPS.
Take note of that last one.
Old school FTP is completely insecure and should never be used.
Whenever you think FTP, think SFTP.
Docker & SSH
If you’re running a Dockerized app on Linode or a similar provider, you likely don’t have a sophisticated orchestration tool such as Kubernetes or Docker Swarm running. Of course this doesn’t mean you can’t, but that tends to be less common for VPS users. So how do we “deploy” a Docker app from CircleCI without manually logging into the server? Let’s look at an example:
deploy: docker: - image: circleci/golang:1.10 steps: - attach_workspace: at: src - add_ssh_keys - run: | docker build . docker login -u $DOCKER_USER -p $DOCKER_PASS docker push myuser/myapp echo 'example.com ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJiGRY6N9WYQ0vy6cTiwAgNbc6ueJmVo/EafBtmT7bcD6cQMbipYM/KfYQ2lCn2TxqWepZKYoyoVQXgArycCOns=' >> ~/.ssh/known_hosts ssh email@example.com <<'ENDSSH' docker pull myuser/myapp docker stop my-app docker rm my-app docker run --name=my-app --restart=always -v $PWD:/app -d myuser/myapp ENDSSH
In this deploy job, we build a new Docker image for our example app and push it to Docker Hub. This makes the app available outside of CircleCI. We then use a feature of the SSH command that allows us to run commands on a remote machine, in this case our production machine. We SSH in, pull the new Docker image onto the production machine, stop, remove, and start the app over with the updated image.
To keep the CircleCI config light, we could also save the remote Docker commands into a Bash script called
update-app.sh or similar, and then run the script from CircleCI.
That would look like this:
ssh firstname.lastname@example.org '/var/app/update-app.sh`
If you give this a try, let me know how you do!
To wrap up, using a VPS to deploy your application is an often-overlooked but viable option if you don’t need the flexible orchestration of larger providers like Kubernetes and AWS S3. My intention with this post is to showcase some of the other deployment options you can use along with CircleCI for shipping applications.