Menu

Continuous Deployment with Google Container Engine

Introduction

Google Cloud Platform (GCP) is a huge, scalable hosting platform that needs little introduction. If you still want one, you can read this overview. In this guide, however, we’ll show you how to continuously deploy a Rails application to Google Container Engine (GKE). You can find full example project on Github at docker-hello-google.

For general information on continuous deployment with CircleCI, see the Introduction to Continuous Deployment.

Tools & Concepts Overview

GKE

GKE is Google’s service to run Docker containers, all powered by Google’s open-source software Kubernetes (K8s). This guide uses Kubernetes v1.2 and depends on the following features:

  • Container Clusters: GKE offers clusters that simplify a normal K8s cluster. With container clusters, GKE automatically runs a master endpoint and any needed nodes across Google Compute Engine (GCE) virtual machines.

  • Deployments: We’ll be using K8s deployments in this guide. This concept replaces/wraps pods and replication controllers/replica sets. Deployments allow you to manage which pods are running and elegantly upgrade containers.

  • gcloud: gcloud is a command-line tool for GCP. This is pre-installed for use on CircleCI.

  • kubectl: kubectl is a command-line tool for K8s. It will be installed via gcloud.

Google Container Registry (GCR)

For smooth deployment, Docker images must be hosted somewhere. While Docker provides its own free registry called Docker Hub, Google provides a registry for image storage while still directly supporting gcloud, GKE, and K8s.

Setup

Prerequisites

This guide makes the following assumptions:

  1. Your project source code is hosted on a CircleCI compatible repository.
  2. You already have a GCP project registered. Keep the project name handy.
  3. There is an already running cluster on GKE. Keep the cluster name handy.
  4. You’re familiar with Docker.

Project Settings on CircleCI

Environment Variables

This project uses several environment variables (envar). Most are set in circle.yml, but one is set via Project Settings for security reasons. This is the GCP Service Account. This key is generated by Google as a JSON file. Follow the Authentication with Google Cloud Platform guide to learn how to generate and base64 encode the key. Create a new envar with the name “ACCT_AUTH” and use the base64 encoded JSON key from Google as the value.

circle.yml

We’ll cover the relevant snippets of the circle.yml file here. You can find the complete file here.

machine:
  environment:
    PROJECT_NAME: circle-ctl-test
    PROJECT_ID: circle-ctl-test
    CLUSTER_NAME: docker-hello-google-cluster
    CLOUDSDK_COMPUTE_ZONE: us-central1-f

In the machine section we set some envars for use to use with gcloud later. You should replace the values of these three variables for your specific project. Note: the project name is actually your project’s ID in Google Cloud.

dependencies:
  pre:
    - sudo /opt/google-cloud-sdk/bin/gcloud --quiet components update --version 120.0.0
    - sudo /opt/google-cloud-sdk/bin/gcloud --quiet components update --version 120.0.0 kubectl
    - echo $ACCT_AUTH | base64 --decode > ${HOME}//gcloud-service-key.json
    - sudo /opt/google-cloud-sdk/bin/gcloud auth activate-service-account --key-file ${HOME}/gcloud-service-key.json
    - sudo /opt/google-cloud-sdk/bin/gcloud config set project $PROJECT_ID
    - sudo /opt/google-cloud-sdk/bin/gcloud --quiet config set container/cluster $CLUSTER_NAME
    - sudo /opt/google-cloud-sdk/bin/gcloud config set compute/zone ${CLOUDSDK_COMPUTE_ZONE}
    - sudo /opt/google-cloud-sdk/bin/gcloud --quiet container clusters get-credentials $CLUSTER_NAME
    # ...

In the above YAML snippet, kubectl is installed via gcloud so the cluster can be changed directly. The JSON key is used to authenticate with GCP, settings are configured, and finally gcloud is used to download cluster credentials so kubectl can use them during the deployment phase.

To ensure that there are no authentication issues with your service key, consider running the above gcloud commands locally.

dependencies:
  pre:
    #...
    - docker build -t us.gcr.io/${PROJECT_NAME}/hello:$CIRCLE_SHA1 .
    - docker tag us.gcr.io/${PROJECT_NAME}/hello:$CIRCLE_SHA1 us.gcr.io/${PROJECT_NAME}/hello:latest

The Docker image for this example Rails app is build using the Git commit hash as a tag. A common practice. The second line also applies the ‘latest’ tag to the image for convenience.

deployment:
  prod:
    branch: master
    commands:
      - ./deploy.sh

Finally, if the build passes and the current branch was the master branch, we run deploy.sh to do the actual deployment work.

deploy.sh

Our deployment script for this repository consist of pushing the newly created Docker image out to the registry, then updating the K8s deployment to use the new image. The following snippets walk through the process. The full deploy.sh file can be found on GitHub.

sudo /opt/google-cloud-sdk/bin/gcloud docker -- push us.gcr.io/${PROJECT_NAME}/hello

Typically when you push a Docker image to a registry, you use the docker push command. While we can still do that here, this gcloud command provides a convenient wrapper that will handle authentication and push the image all at once. Our new image is now available in GCR for all our GCP infrastructure to access.

sudo chown -R ubuntu:ubuntu /home/ubuntu/.kube

Since we used gcloud as root (via sudo), the newly installed kubectl binary doesn’t have the right permissions. So we fix it. We’re looking to simplify the use of gcloud and kubectl in the future.

kubectl patch deployment docker-hello-google -p '{"spec":{"template":{"spec":{"containers":[{"name":"docker-hello-google","image":"us.gcr.io/circle-ctl-test/hello:'"$CIRCLE_SHA1"'"}]}}}}'

This command utilizes the patch subcommand of kubectl to make a change to our running deployment on the fly. This is extremely useful in a CI environment. Normally you might use kubectl edit deployment which will open your deployment spec file in your default text editor.

This command finds the line that specifies the image to use for our container, and replaces it with the image tag of the image just built. The K8s deployment then intelligently upgrades the cluster by shutting down old containers and starting up-to-date ones.

Summary & Notes

We’ve seen how we can use CircleCI to deploy an app, in this examples, a Rails app, to Google Container Engine while using Google Container Registry to store the images. An example of a green (passing) build for the example project can be found here.

The Example Project Doesn’t Run on GKE

If you were following along with the example project on GitHub, you may have noticed that the Rails app doesn’t run on GKE. This is because the environment variable ‘SECRET_KEY_BASE’ is needed. In circle.yml this is set for the test to pass. This also needs to be set in your GKE deployment. There is more than one way to do this. For this project, the deployment spec was manually edited to set that variable using the command kubectl edit deployment.

How To Setup The Project and Cluster.

This is best left to GCP and Kubernetes docs: