Docs
circleci.com
Start Building for Free

CircleCI Self-hosted Runner Installation on Kubernetes

1 week ago2 min read
Cloud
Server v4.x
Server v3.x
On This Page

Introduction

This installation guide is to help set up CircleCI self-hosted runners on your Kubernetes cluster. We provide a Helm chart to simplify the installation process.

The Helm chart will spin up one or more pods of the same self-hosted runner resource class. This is useful for when you want all of these self-hosted runners to execute jobs requesting the same execution environment. Each runner will pull jobs off the queue on an as-available basis.

If you want to have different self-hosted runners specialized for different workloads, it is recommended to create different self-hosted runner resource classes and rerun these instructions and have separate charts for each self-hosted runner class you create.

If you are using Server, please make sure you read CircleCI server installation.

Prerequisites

Getting Started

  1. Clone the repository at https://github.com/CircleCI-Public/circleci-runner-k8s.

  2. Modify the chart’s values in values.yaml per your needs. See documentation on the list of values.

  3. Apply your chart to your cluster, setting the required parameters you should have obtained from the authentication step:

    $ helm install "circleci-runner" ./ \
      --set runnerToken=$CIRCLECI_RUNNER_TOKEN \
      --set resourceClass=$CIRCLECI_RUNNER_RESOURCE_CLASS \
      --namespace your-namespace

    For further information about configuring the helm chart see the readme

  4. Verify your pods are up and running by checking their status and following the logs. You should see output like the following:

    $ kubectl get pods --all-namespaces
    NAMESPACE     NAME                                                             READY   STATUS    RESTARTS   AGE
    default       circleci-runner-test-7d6b8fc6f-4z5wl                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-h97jz                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-pksc6                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-q74p4                             1/1     Running   0          28h
    default       circleci-runner-test-7d6b8fc6f-wh6m2                             1/1     Running   0          28h
    
    $ kubectl logs -f circleci-runner-test-7d6b8fc6f-4z5wl
    
    ... (output truncated)
    
    time="2021-03-25T20:55:40Z" level=info msg="CircleCI launcher starting" config=/var/opt/circleci/launch-agent-config.yaml
    time="2021-03-25T20:55:40Z" level=info msg="loaded config" name=circleci-runner-test-7d6b8fc6f-4z5wl url="https://runner.circleci.com"
    time="2021-03-25T20:55:40Z" level=info msg="no task found"

Chart Values

Customizable parameters are left inside the values.yaml file. See the following chart for information about each of the values:

ValueDefaultRequired?Description

image.repository image.tag

circleci/runner launch-agent

Y

You can extend a custom Docker image from the CircleCI default self-hosted runner and use that instead.

For CircleCI Enterprise installations, you can use the image.tag to set a pinned version of launch agent. See the Self-hosted Runner Installation page for the compatible version tags.

replicaCount

1

Y

The number of replicas of this self-hosted runner you want in your cluster. Must currently be set and updated manually. See Pending Work.

resourceClass

-

Y

The resource class you created for your self-hosted runner. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.

runnerToken

-

Y

The token you created for your self-hosted runner resource class. You can choose to fill it in the chart here or to pass it directly when applying the chart, as shown above.

env

-

N

Environment variables to set in the launch-agent pod. Including values for runner configuration

all other values

-

N

Modify at your own discretion and risk.

CircleCI server installation

When installing the Helm chart for use with a CircleCI server installation, the image.tag will need to be set to the pinned launch agent version specified in the Self-hosted Runner Installation instructions. The LAUNCH_AGENT_API_URL will also need to be set as an environment variable. This can be done with the --set flag, or in the env section of the values.yaml file, specifying the hostname or address of the server installation.

Upgrading self-hosted runner deployment for server

  1. Modify the values.yaml file to specify the new image.tag to update to. Refer to the Chart Values section of this document for more details about the values.yaml file.

  2. Deploy the changes to the cluster

    $ helm upgrade -f values.yaml "circleci-runner" ./ \
      --set runnerToken=$CIRCLECI_RUNNER_TOKEN \
      --set resourceClass=$CIRCLECI_RUNNER_RESOURCE_CLASS \
      --set env.LAUNCH_AGENT_API_URL=<server_host> \
      --namespace your-namespace

    Further information about the $ helm upgrade command and its usage can be found in the helm documentation

Limitations and pending work

  • The Helm chart currently does not configure permissions, therefore, the containers will not have elevated privileges. If you need elevated permissions (e.g., for Docker in Docker work, etc.), you will need to modify the chart yourself.

  • The Helm chart currently does not support auto-scaling - you will need to modify the replicaCount parameter inside values.yaml yourself and re-apply the chart to your cluster.

  • The Helm chart currently expects only one self-hosted runner resource class type and one token. If you want to configure clusters for multiple self-hosted runner resource classes, you will need to set up separate charts for each one.


Help make this document better

This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.

Need support?

Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.

You can also visit our support site to find support articles, community forums, and training resources.