CircleCI Runner on Kubernetes

Introduction

CircleCI runner is available on the Scale Plan and for server customers running server v3.1.0 and up. Please reach out to your sales representative (or contact us) for information on how to sign up for the Scale plan.

This installation guide is to help set up CircleCI runner on your Kubernetes cluster. We provide a Helm chart to simplify the installation process.

The Helm chart will spin up one or more pods of the same runner resource class. This is useful for when you want all of these runners to execute jobs requesting the same execution environment. Each runner will pull jobs off the queue on an as-available basis.

If you want to have different runners specialized for different workloads, it is recommended to create different runner resource classes and rerun these instructions/have separate charts for each runner class you create.

Prerequisites

CircleCI Server Installation

When installing the Helm chart for use with a CircleCI server installation the agentVersion will need to be set to the pinned version specified in the Runner Installation instructions.

Upgrading Runner Deployment for Server

  1. Modify the values.yaml file to specify the new agentVersion to update to. Refer to the Chart Values section of this document for more details about the values.yaml file.

  2. Deploy the changes to the cluster

    $ helm upgrade -f values.yaml "circleci-runner" ./ \
      --set runnerToken=$CIRCLECI_RUNNER_TOKEN \
      --set resourceClass=$CIRCLECI_RUNNER_RESOURCE_CLASS \
      --namespace your-namespace

    Further information about the $ helm upgrade command and its usage can be found in the helm documentation

Getting Started

  1. Clone the repository at https://github.com/CircleCI-Public/circleci-runner-k8s.

  2. Modify the chart’s values in values.yaml per your needs. See documentation on the list of values.

  3. Apply your chart to your cluster, setting the required parameters you should have obtained from the authentication step:

    $ helm install "circleci-runner" ./ \
      --set runnerToken=$CIRCLECI_RUNNER_TOKEN \
      --set resourceClass=$CIRCLECI_RUNNER_RESOURCE_CLASS \
      --namespace your-namespace
  4. Verify your pods are up and running by checking their status and following the logs. You should see output like the following:

    $ kubectl get pods --all-namespaces
    NAMESPACE     NAME                                                             READY   STATUS    RESTARTS   AGE
    default       circleci-runner-test-7d6b8fc6f-4z5wl                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-h97jz                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-pksc6                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-q74p4                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-wh6m2                             1/1     Running   0          28h
    
    $ kubectl logs -f circleci-runner-test-7d6b8fc6f-4z5wl
    
    ... (output truncated)
    
    time="2021-03-25T20:55:40Z" level=info msg="CircleCI launcher starting" config=/opt/circleci/launch-agent-config.yaml
    time="2021-03-25T20:55:40Z" level=info msg="loaded config" name=circleci-runner-test-7d6b8fc6f-4z5wl url="https://runner.circleci.com"
    time="2021-03-25T20:55:40Z" level=info msg="no task found"

Chart Values

Customizable parameters are left inside the values.yaml file. See the following chart for information about each of the values:

Value Default Required? Description

image.repository image.tag

circleci/runner launch-agent

Y

You can extend a custom Docker image from the CircleCI default runner and use that instead.

replicaCount

1

Y

The number of replicas of this runner you want in your cluster. Must currently be set and updated manually. See Pending Work.

resourceClass

-

Y

The resource class you created for your runner. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.

runnerToken

-

Y

The token you created for your runner resource class. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.

agentVersion

-

N

The circleci-task-agent version to pin. This is only used for CircleCI Server installations.

all other values

-

N

Modify at your own discretion and risk.

Limitations/Pending Work

  • The Helm chart currently does not configure permissions, therefore, the containers will not have elevated privileges. If you need elevated permissions (e.g., for Docker in Docker work, etc.), you will need to modify the chart yourself.

  • The Helm chart currently does not support auto-scaling - you will need to modify the replicaCount parameter inside values.yaml yourself and re-apply the chart to your cluster.

  • The Helm chart currently expects only one runner resource class type and one token. If you want to configure clusters for multiple runner resource classes, you will need to set up separate charts for each one.



Help make this document better

This guide, as well as the rest of our docs, are open-source and available on GitHub. We welcome your contributions.