Deploy to Amazon EKS using CircleCI and Helm

Kubernetes is a container orchestration system for automating deployment and managing containerized applications. Helm is a Kubernetes package manager that helps you define, install, and upgrade Kubernetes applications. It lets you define reusable templates for Kubernetes components (deployment, service, hpa, service account, etc.) that can be published and shared across applications.

In this tutorial, you will learn how to build and install Helm charts for your application to an AWS EKS cluster. We will create and Dockerize a Python Flask app, then create Helm charts for it. Finally you will automate its deployment to an existing AWS EKS cluster using CircleCI.

Prerequisites

For this tutorial, you will need to set up Python on your system so that you can define the Flask app. Also, you need to create an AWS account for deploying the application and a CircleCI account for automating the deployments.

Refer to this list to set up everything required for this tutorial.

• Install Python 3
• Install Helm CLI for building and installing Helm Charts.
• Create an AWS account and a CircleCI account

During development, if you would like to test the set up locally, you will need to set up a few additional things:

• Install Docker for building Docker images locally.
• Install kubectl for testing the cluster locally.
• Install AWS CLI and configure AWS credentials .
• Install eksctl for managing Kubernetes clusters on Amazon EKS.

Our tutorials are platform-agnostic, but use CircleCI as an example. If you don’t have a CircleCI account, sign up for a free one here.

Defining a Flask app

In this section, you will create a sample Flask app using Python 3 and containerize it using Docker. Flask is an open source Python framework for developing web applications. It is a popular choice for building the API service layer for lightweight applications.

Creating the app

First, create a new directory for your Flask project and go to it. Enter:

mkdir flask-helm-charts-circleci
cd flask-helm-charts-circleci

Next, create a requirements.txt file in the flask-helm-charts-circleci directory for defining the app’s library dependencies. The Python app depends on the flask package, so add it in the requirements.txt file.

Flask==2.2.2

Create a app.py file in the flask-helm-charts-circleci directory and add this to it.

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return 'Hello World!'

if __name__ == '__main__':
    app.run(debug=False, host='0.0.0.0')

This code snippet defines a GET route that returns a static response. Based on your use case, you can add custom logic in the hello_world method. Also, you can add more routes like this:

@app.route('/health')
def health_check():
    return 'Health OK'

You can run the app locally by executing this command. On executing this command, the Flask app will start running on localhost port 5000.

python app.py

Go to http://127.0.0.1:5000 to verify the response.

Dockerizing the app

Now that you have defined the Flask app, add a Dockerfile to containerize the app. We will use Python 3.9.1 base image from Docker hub. Create a Dockerfile in the flask-helm-charts-circleci directory and add this code snippet to it.

FROM python:3.9.1

ADD . /app
WORKDIR /app
RUN pip install -r requirements.txt

EXPOSE 5000

CMD ["python", "app.py"]

This code snippet has these steps:

• It uses the Python 3.9.1 base image.
• The ADD . step copies all the files from the root directory of the project to the /app directory on the container.
• WORKDIR sets the current working directory of the container to /app. All the subsequent commands will be executed in this directory.
• RUN step installs the python dependencies defined in the requirements.txt file.
• EXPOSE step exposes port 5000 from the Docker container.
• Finally, CMD defines the container’s startup command. The container will start the Flask app when it is executed.

Building and running the Docker container

Build and run the Docker container locally to verify if it works as expected. If you plan to use CircleCI for your set up, you can skip this step.

To build the Docker image, run this command in the flask-helm-charts-circleci directory.

docker build -t flask-hello-world .

Note: Make sure to have Docker installed on your system before executing the above command. Follow the link in the Prerequisites section for installation steps.

Finally, run this command.

docker run -it --rm -p 5001:5000 flask-hello-world:latest

This command forwards port 5000 from the Docker container to port 5001 locally. Try navigating to http://127.0.0.1:5001 to verify that the app is running as expected.

Creating AWS resources

Before moving on to testing and deployment, make sure that you have the required AWS resources provisioned in your account. You could use existing resources in your account or provision new resources if you don’t have them created.

• Create a AWS ECR repository for hosting your Flask Docker images.
• Create a AWS ECR repository for hosting your Helm chart.
• Create an AWS ECR cluster for running your Kubernetes workload.

Make sure that you use the same IAM user for creating the cluster and configuring the credentials on CircleCI.

Creating Helm charts

Charts are Helm’s packaging format. A chart contains a collection of files that describe a related set of Kubernetes resources. In this section, you will define a Helm chart for your app.

To create a chart for your app, execute this command in the flask-helm-charts-circleci directory.

helm create charts

Note: Install the Helm CLI before executing the previous command. Follow the link in the Prerequisites section for installation steps.

Executing this command will create a charts directory with this structure.

charts/
  Chart.yaml
  values.yaml
  charts/
  templates/

The charts/templates/ directory contains several files but you need only _helpers.tpl, deployment.yaml, and service.yaml. Delete the other files from the templates directory.

Next, update the deployment template to remove some variables which are not needed for this tutorial. You also need to update the containerPort to 5000 since your Flask application is running on this port. Replace the contents of the charts/templates/deployment.yaml with this code snippet:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "charts.fullname" . }}
  labels:
    {{- include "charts.labels" . | nindent 4 }}
spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
  {{- end }}
  selector:
    matchLabels:
      {{- include "charts.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      {{- with .Values.podAnnotations }}
      annotations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      labels:
        {{- include "charts.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: http
              containerPort: 5000
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /
              port: http
          readinessProbe:
            httpGet:
              path: /
              port: http

Next, update the values.yaml file with values corresponding to your application. Replace the contents of the charts/values.yaml with this code snippet:

replicaCount: 1

image:
  repository: <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com/<AWS_ECR_REPO_NAME>
  pullPolicy: IfNotPresent
  tag: "latest"

labels:
  env: "staging"

service:
  port: 5000
  targetPort: 5000
  type: LoadBalancer

replicaCount: 1

autoscaling:
  enabled: false

ingress:
  enabled: false

imagePullSecrets:
  - name: regcred

Replace <AWS_ACCOUNT_ID> with your AWS account ID, <AWS_REGION> with the AWS deployment region and <AWS_ECR_REPO_NAME> with the name of the AWS ECR repository for your Flask Docker image.

Now, update the name in the charts/Chart.yaml.

name: flask-helm-chart

The chart name should match the name of the AWS ECR Helm repository.

Testing locally

You may want to test the setup locally, before automating the deployments using CircleCI. This section describes how to manually push the image and Helm chart to AWS ECR and deploy the Helm chart to the AWS EKS cluster. If you want to directly automate the deployments using CircleCI, you can skip this section.

Pushing the Docker image to ECR

Before pushing the Docker image you need to authenticate Docker with AWS ECR. Execute this command for authentication:

aws ecr get-login-password --region <AWS_REGION> | docker login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com

You need AWS CLI installed and AWS credentials configured before executing the previous command. Follow the links in the Prerequisites section for instructions.

Next, tag the Docker image you built in the previous section using this command:

docker tag flask-hello-world:latest <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com/flask-hello-world

Make sure that your AWS account contains a ECR repository named flask-hello-world.

Push the Docker image to the AWS ECR repository by executing this command:

docker push <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com/flask-hello-world:latest

Pushing the Helm chart to ECR

Before pushing the Helm chart you need to authenticate helm with AWS ECR. Execute this command for authentication:

aws ecr get-login-password --region us-west-2 | helm registry login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com

Next, package the Helm chart by executing the this command from the root directory of the project.

helm package charts

It will generate a flask-helm-chart-0.1.0.tgz file with the compressed version of your helm chart.

The tgz filename could differ based on the name of the chart defined in charts/Chart.yaml.

Next, you can push the Helm chart to AWS ECR repository by executing this command:

helm push flask-helm-chart-0.1.0.tgz oci://<AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com/

Deploying the Helm chart to AWS EKS

Now, that your Helm chart is pushed to the AWS ECR repository, you can deploy it to the AWS EKS cluster. First, update the local kube config using eksctl.

eksctl utils write-kubeconfig --cluster=<AWS_EKS_CLUSTER_NAME> --region=<AWS_REGION>

Make sure that eksctl is installed on your system. Replace <AWS_EKS_CLUSTER_NAME> with the name of your AWS EKS cluster and <AWS_REGION> with the name of the selected AWS region.

This command will write the kube config to $HOME/.kube/config.

Finally, execute the helm install command to install the chart to the EKS cluster.

helm install flask-helm oci://<AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com/flask-helm-chart --version 0.1.0

The command will install the workload to your EKS cluster. You can verify the same by checking the pods on your cluster.

kubectl get pods

Automate deployment using CircleCI

Now that you have tested the Kubernetes cluster locally, automate the workflow so that the changes can be deployed every time you push code to the main branch. To automate the deployments you need to:

• Add Helm deployment script
• Add the configuration script
• Set up the project on CircleCI

Add Helm deployment script

When an existing Helm chart on a cluster is updated, you need to use the helm upgrade command instead of helm install. You can define a shell script that checks whether a chart with the name flask-helm is already deployed on the cluster and whether it uses the appropriate command. Create a file at scripts/deploy-flask.sh and add this to it.

#!/bin/bash
TAG=$1
AWS_ECR_ACCOUNT_URL=$2
AWS_ECR_HELM_REPO_NAME=$3
echo "oci://$AWS_ECR_ACCOUNT_URL/$AWS_ECR_HELM_REPO_NAME"
export KUBECONFIG=$HOME/.kube/config
result=$(eval helm ls | grep flask-helm)
if [ $? -ne "0" ]; then
   helm install flask-helm "oci://$AWS_ECR_ACCOUNT_URL/$AWS_ECR_HELM_REPO_NAME" --version $TAG
else
   helm upgrade flask-helm "oci://$AWS_ECR_ACCOUNT_URL/$AWS_ECR_HELM_REPO_NAME" --version $TAG
fi

The shell script will receive TAG, AWS_ECR_ACCOUNT_URL and AWS_ECR_HELM_REPO_NAME as parameters.

Add the configuration script

First, add a .circleci/config.yaml script in the project’s root containing the configuration file for the CI pipeline. Add this code snippet to it.

version: 2.1

orbs:
  aws-ecr: orbies/aws-ecr@1.2.1

jobs:
  install_helm_chart:
    docker:
      - image: cimg/python:3.10.7
    steps:
      - checkout
      - run:
          name: Install awscli
          command: curl --silent "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && unzip awscliv2.zip && sudo ./aws/install
      - run:
          name: Install eksctl
          command: curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp && sudo mv /tmp/eksctl /usr/local/bin
      - run:
          name: Install and configure kubectl
          command: curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" && sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl && kubectl version --client
      - run:
          name: Install and configure kubectl aws-iam-authenticator
          command: curl -Lo aws-iam-authenticator https://github.com/kubernetes-sigs/aws-iam-authenticator/releases/download/v0.5.9/aws-iam-authenticator_0.5.9_linux_amd64 && chmod +x ./aws-iam-authenticator && mkdir -p $HOME/bin && cp ./aws-iam-authenticator $HOME/bin/aws-iam-authenticator && export PATH=$PATH:$HOME/bin
      - run:
          name: Install and configure helm
          command: sudo curl -L https://get.helm.sh/helm-v3.10.1-linux-amd64.tar.gz | tar xz && sudo mv linux-amd64/helm /bin/helm && sudo rm -rf linux-amd64
      - run:
          name: "docker login"
          command: |
            aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ECR_ACCOUNT_URL
      - run:
          name: "helm login"
          command: |
            aws ecr get-login-password --region $AWS_DEFAULT_REGION | helm registry login --username AWS --password-stdin $AWS_ECR_ACCOUNT_URL
      - run:
          name: "cluster configs"
          command: |
            eksctl utils write-kubeconfig --cluster=$AWS_EKS_CLUSTER_NAME --region=$AWS_CLUSTER_REGION
      - run:
          name: "helm install"
          command: bash ./scripts/deploy-flask.sh 1.0.0 $AWS_ECR_ACCOUNT_URL $AWS_ECR_HELM_REPO_NAME

workflows:
  build_and_push_image:
    jobs:
      - aws-ecr/build-and-push-image:
          aws-access-key-id: AWS_ACCESS_KEY_ID
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY
          repo: "${AWS_ECR_REPO_NAME}"
          docker-login: false
          account-url: AWS_ECR_ACCOUNT_URL
          region: AWS_DEFAULT_REGION
          tag: "latest"
      - aws-ecr/push-helm-chart:
          account-url: AWS_ECR_ACCOUNT_URL
          aws-access-key-id: AWS_ACCESS_KEY_ID
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY
          create-repo: false
          path: ./charts
          region: AWS_DEFAULT_REGION
          repo: "${AWS_ECR_HELM_REPO_NAME}"
          tag: 1.0.0
          requires:
            - aws-ecr/build-and-push-image
      - install_helm_chart:
          requires:
            - aws-ecr/push-helm-chart

Take a moment to review the CircleCI configuration. The config defines three jobs:

• The aws-ecr/build-and-push-image job uses the aws-ecr orb to build and push the Flask docker image to the AWS ECR repository.
• The aws-ecr/push-helm-chart job also uses the aws-ecr orb to package and push the Helm chart to the AWS ECR Helm repository.
• The install_helm_chart pulls the Helm chart from the ECR repository and installs it on the AWS EKS cluster. It has these steps:
  • Install required tools like awscli, eksctl, kubectl, aws-iam-authenticator and helm.
  • Use AWS CLI to authenticate Docker and Helm with AWS ECR.
  • Use eksctl to write the kube configs to $HOME/.kube/config.
  • Install the Helm chart to the EKS cluster using the scripts/deploy-flask.sh script.

Now that the configuration file has been properly set up, create a repository for the project on GitHub and push all the code to it. Review Pushing a project to GitHub for instructions.

Setting up the project on CircleCI

Next, log in to your CircleCI account. On the CircleCI dashboard, click the Projects tab, search for the GitHub repo name and click Set Up Project for your project.

You will be prompted to add a new configuration file manually or use an existing one. Since you have already pushed the required configuration file to the codebase, select the Fastest option and enter the name of the branch hosting your configuration file. Click Set Up Project to continue.

Completing the setup will trigger the pipeline. The pipeline will fail in its first run since you haven’t defined the environment variables.

Setup Environment Variables

Click Project settings from the project dashboard and go to the Environment variables tab. Click the Add environment variable button to add a new key value.

Add these environment variables:

• AWS_ACCESS_KEY_ID to the access key obtained while creating AWS credentials.
• AWS_SECRET_ACCESS_KEY to the secret obtained while creating AWS credentials.
• AWS_DEFAULT_REGION to a region where you wish to deploy your application.
• AWS_ECR_ACCOUNT_URL to your AWS account URL. Replace the values for AWS account ID and region in <AWS_ACCOUNT_ID>.dkr.ecr.<AWS_REGION>.amazonaws.com and set it as the account URL.
• AWS_ECR_HELM_REPO_NAME to the name of AWS ECR repository for Helm charts.
• AWS_ECR_REPO_NAME to the name of AWS ECR repository for your Flask application’s Docker image.
• AWS_EKS_CLUSTER_NAME to the name of your AWS EKS cluster.
• AWS_CLUSTER_REGION The region where your cluster was created. This might be different or the same as the default region.

Once the environment variables are configured, rerun the pipeline. This time it should build successfully.

Conclusion

In this tutorial, you learned how to automatically build and deploy Helm charts to an AWS EKS cluster using CircleCI. Helm charts make it easier to handle the complexities of the cluster with reusable templates and simplify the process of updating a cluster. Also, it is easy to version, share, and host the Helm charts on public or private servers. With CircleCI, you can automate the build and deployment pipeline for continuous delivery. The pipeline can be used to push Docker images and Helm charts to AWS ECR repositories and update the EKS cluster with the new workload.

You can check out the complete source code used in this tutorial on GitHub.

---

Vivek Kumar Maskara is a Software Engineer at JP Morgan. He loves writing code, developing apps, creating websites, and writing technical blogs about his experiences. His profile and contact information can be found at maskaravivek.com.

Read more posts by Vivek Maskara