Docs
Start Building for Free
CircleCI.comAcademyBlogCommunitySupport

Machine runner 3.0 configuration reference - Open preview

1 week ago5 min read
Cloud
On This Page

A YAML file is used to configure the machine runner, how it communicates with CircleCI’s servers, and how it will launch the task-agent. This page explains all available parameters.

Introduction

The configuration file uses the following format with the various parameters explained in more detail below, and is by default located at /etc/circleci-runner/circleci-runner-config.yaml on Linux machines.

api:
  auth_token: << AUTH TOKEN >>
runner:
  name: runner-name-001
  working_directory: "/var/lib/circleci-runner/workdir"
  cleanup_working_directory: true
logging:
  level: "info"
  format: "text"
  file: "-"

api.auth_token

$CIRCLECI_RUNNER_API_AUTH_TOKEN

This is a token used to identify the machine runner to CircleCI and can be generated by the CircleCI CLI. An existing token may be shared among many installations, but this token only allows a particular resource_class to be specified.

api.url

$CIRCLECI_RUNNER_API_URL

This is a URL pointing to https://runner.circleci.com by default. It should not be modified unless you have a CircleCI server installation. In that case, it must be set to the URL of your server installation.

runner.name

$CIRCLECI_RUNNER_NAME

RUNNER_NAME is a unique name of your choosing assigned to this particular machine runner. CircleCI recommends using the hostname of the machine so that it can be used to identify the agent when viewing statuses and job results in the UI on the CircleCI web app. The only special characters accepted in RUNNER_NAME are . () - _.

logging.level

$CIRCLECI_RUNNER_LOGGING_LEVEL

This sets the logging level for the circleci-runner service. The logging level defaults to info.

Example:

logging:
  level: "info"

logging.format

$CIRCLECI_RUNNER_LOGGING_FORMAT

This sets the logging format for the circleci-runner service used for stderr and file logging. The logging format defaults to color.

Example:

logging:
  format: "color"

logging.file

$CIRCLECI_RUNNER_LOGGING_FILE

This controls the file where the machine runner launch-agent and task-agent logs will go. See Runner concepts for more information on launch-agents and task-agents.

Example:

logging:
  file: "-"

runner.task-agent-directory

$CIRCLECI_RUNNER_TASK_AGENT_DIRECTORY

This parameter accepts the path to the directory of the circleci-runner task agent binary and defaults to ~/.local/bin.

runner:
  task-agent-directory: "~/.local/bin"

runner.command_prefix

$CIRCLECI_RUNNER_COMMAND_PREFIX

This prefix takes a YAML list of arguments that wraps and runs the task-agent.

For example, sudo can elevate permissions:

runner:
  command_prefix: ["sudo", "-niHu", "USERNAME", "--"]

The prefix can also be a custom script. The arguments passed to the script will launch the task-agent.

The custom script must:

  • Take great care to ensure the task-agent (that is, the arguments passed to it) runs

  • Forward the exit code from task-agent as its own exit code

Example script saved as /var/opt/circleci/wrapper.sh:

#!/bin/bash

task_agent_cmd=${@:1}
echo "About to run CircleCI task agent: ${task_agent_cmd}"
$task_agent_cmd
exit=$?
echo "CircleCI task agent finished."
exit $exit

Here is the configuration snippet for the example above:

runner:
  command_prefix: ["/var/opt/circleci/wrapper.sh"]

runner.working_directory

$CIRCLECI_RUNNER_WORK_DIR

This directory takes a fully qualified path and allows you to control the default working directory used by each job. If the directory already exists, the task-agent will need permissions to write to the directory. If the directory does not exist, then the task-agent will need permissions to create the directory. If installing multiple launch-agents on the same machine, each launch-agent will need a unique working directory.

Example:

runner:
  working_directory: "/var/lib/circleci-runner/workdir"

runner.cleanup_working_directory

$CIRCLECI_RUNNER_CLEANUP_WORK_DIR

This flag enables you to control the working directory cleanup after each job.

The possible values are:

  • true

  • false

Example:

runner:
  cleanup_working_directory: true

runner.use_home_ssh_dir_for_checkout_keys

$USE_HOME_SSH_DIR_FOR_CHECKOUT_KEYS

This flag enables you to use the home directory of the user running the self-hosted runner instance for storing SSH checkout keys.

The possible values are:

  • true

  • false

Example:

runner:
   use_home_ssh_dir_for_checkout_keys: true

runner.mode

$CIRCLECI_RUNNER_MODE

This parameter allows you to specify whether you want to terminate this self-hosted runner instance upon completion of a job (single-task), or to continuously poll for new available jobs (continuous).

The possible values are:

  • continuous

  • single-task

Example:

runner:
  mode: continuous

runner.max_run_time

$CIRCLECI_RUNNER_MAX_RUN_TIME

This value can be used to override the default maximum duration the task-agent will run each job. Note that the value is a string with the following unit identifiers h, m or s for hour, minute, and seconds respectively:

Here are a few valid examples:

  • 72h - 3 days

  • 1h30m - 1 hour 30 minutes

  • 30s - 30 seconds

  • 50m - 50 minutes

  • 1h30m20s - An overly specific (yet still valid) duration

Example:

runner:
  max_run_time: 5h

Customizing job timeouts and drain timeouts

If you would like to customize the job timeout setting, you can “drain” the job by sending the machine runner a termination (TERM) signal, which then causes the machine runner to attempt to gracefully shutdown. When this TERM signal is received, the machine runner enters draining mode, preventing the machine runner from accepting any new jobs, but still allowing any current active job to be completed. At the end of draining, the machine runner then signals the task-agent to cancel any active job (by sending it a TERM signal).

Draining can end in one of two ways:

  • The task has been in the draining state for longer than the configured max_run_time

  • An additional TERM signal is received by the machine runner during draining

runner.idle_timeout

$CIRCLECI_RUNNER_IDLE_TIMEOUT

This timeout will enable a machine runner to terminate if no task has been claimed within the given time period. The value is a string with the following unit identifiers: h, m or s for hours, minutes, and seconds respectively (for example, 5m is 5 minutes).

Example:

runner:
  idle_timeout: 1h

runner.ssh.advertise_addr

$CIRCLECI_RUNNER_SSH_ADVERTISE_ADDR

This parameter enables the “Rerun job with SSH” feature. Before enabling this feature, there are important considerations that should be made. Rerun with SSH is not currently available on container runner.

The address is of the form host:port and is displayed in the “Enable SSH” and “Wait for SSH” sections for a job that is rerun.

Example:

runner:
  ssh:
    advertise_addr: HOSTNAME:54782

Considerations before enabling SSH debugging

Task-agent runs an embedded SSH server and agent on a dedicated port when the “Rerun job with SSH” option is activated. This feature will not affect any other SSH servers or agents on the system that the self-hosted runner is installed on.

  • The host port used by the SSH server is currently fixed to 54782. Ensure this port is unblocked and available for SSH connections. A port conflict can occur if multiple machine runners are installed on the same host.

  • The SSH server will inherit the same user privileges and associated access authorizations as the task-agent, defined by the runner.command_prefix parameter.

  • The SSH server is configured for public key authentication. Anyone with permission to initiate a job can rerun it with SSH. However, only the user who initiated the rerun will have their SSH public keys added to the server for the duration of the SSH session.

  • Rerunning a job with SSH will hold the job open for two hours if a connection is made to the SSH server, or ten minutes if no connection is made, unless cancelled. While in this state, the job is counted against an organization’s concurrency limit, and the task-agent will be unavailable to handle other jobs. Therefore, it is recommended to cancel an SSH rerun job explicitly (through the web UI or CLI) when finished debugging.

Basic full configuration for a machine runner

The fields you must set for a specific job to run using your self-hosted runners are:

  • machine: true

  • resource_class: <namespace>/<resource-class>

Simple example of how you could set up a job:

version: 2.1

workflows:
  build-workflow:
    jobs:
      - runner
jobs:
  runner:
    machine: true
    resource_class: <namespace>/<resource-class>
    steps:
      - run: echo "Hi I'm on Runners!"

The job will then execute using your self-hosted runner when you push the .circleci/config.yml to your VCS provider.

Suggest an edit to this page

Make a contribution
Learn how to contribute

Still need help?

Ask the CircleCI community
Join the research community
Visit our Support site