Runner Release 3.1.7

Both Machine and Container Runner:

Addressed CVEs in dependencies of the runner agent binary.

Container Runner:

The orchestrator init container’s resource requirements are now configurable via orchestrator.resources in the Helm values (chart version v101.1.6+).

Runner Release 3.1.6

Container Runner:

• Fixed compatibility issues between logging collector updates and older Helm chart versions. The container-agent-instance label in the Pod informer is now optional, preventing scheduling conflicts when container agent instances run with mismatched component versions.

Runner Release 3.1.5

Container Runner:

• Version 3.1.5 introduces Windows support for container runner in open preview. For detailed setup instructions and configuration guidance, visit our Windows container runner documentation.
• Container runner now filters for pods specifically managed by each replica, rather than watching all task pods with the managed-by: circleci-container-agent label. This optimization reduces API load when running multiple container agent instances within the same cluster.
• Added support for running the primary container under a custom user ID (UID). You can now specify a valid UID using the docker.user key in your CircleCI configuration, which will be mapped to the task pod container user.

Reminder

Maintenance window for Runner is scheduled for June 3rd, 2025, at 19:00 PST/22:00 EST. The maintenance window will last until 19:10 PST/22:10 EST.

During this period:

• Resource management will not be available
• The Runner web UI for inventory and installation will not be available
• Up to a 5-minute delayed start time for runner jobs.

Maintenance window for Runner is scheduled for June 3rd, 2025, at 19:00 PST/22:00 EST. The maintenance window will last until 19:10 PST/22:10 EST.

During this period:

• Resource management will not be available
• The Runner web UI for inventory and installation will not be available
• Up to a 5-minute delayed start time for runner jobs.

Runner Release 3.1.3

Machine Runner:

• Fixed a bug where an error downloading the task agent binary was not correctly propagated, causing the runner to get stuck without retrying the download and requiring a manual restart to unblock.

Runner Release 3.1.2

Container Runner:

• Set resource requirements (requests and limits) on the orchestrator init container (https://github.com/circleci/runner-init). This change helps ensure the Pod is schedulable when resource quotas are applied.
• Added options to configure the image name for the orchestrator container. This can be used for hosting the image in a private registry or within an air-gapped environment on CircleCI server. Note this change requires version v101.1.3 of the Helm chart.

Runner Release 3.1.1

Container Runner:

• Fixed the volume for GOAT and task-agent binaries to be read-write. It was unnecessarily read-only and caused issues with the circleci-cli orb, which overwrites the circleci symlink with the CLI.

Runner Release 3.1.0

Container Runner:

• Version 3.1.0 introduces substantial performance and reliability improvements to container runner. These updates are designed to enhance the overall efficiency and stability of operations at scale. For more background and technical details on these changes, you can visit our Runner Init repository and documentation.
• To upgrade, ensure that the Helm chart is at least version v101.1.2. The chart is pinned to the major version tag of the runner, enabling automatic upgrades on redeployment. If you wish to remain on the existing architecture (3.0.x), you can do so by setting the tag to kubernetes-3.0. This version will continue to receive only critical bug fixes and security patches. You can check the configuration details in the Helm chart repository.

Runner Release 3.0.25

Both Machine and Container Runner:

• Sign docker manifests to be verified with the CircleCI Public Key
• Security fixes
• Increase time between deleting existing task-agents to prevent unexpected removals during task execution

Machine Runner:

• Set KillMode=process for linux based machine runners

As previously reported https://circleci.com/changelog/launch-agent-1-x-variants-eol/.

Support for all 1.x variants of runner launch agent has ended. It has been replaced with Machine Runner 3.0. Any remaining access will be removed September 17th, 2024. For those that haven’t migrated, your builds on runner will fail.

You can learn more about machine runner and migration here.

Runner Release 3.0.24

Both Machine and Container Runner:

• Fix a bug which broke OIDC token injection for Cloud Runner users

NOTE: Release 3.0.23 has been removed from our package repository. It contained bug breaking OIDC token injection for Cloud Runner users. Please use this release 3.0.24 instead.

NOTE: Release 3.0.23 has been removed from our package repository. It contained bug breaking OIDC token injection for Cloud Runner users. Please use this release 3.0.24 instead.

Container Runner

• Added support for customizable service containers, allowing users more control over a service’s resource usage on a per-image basis. For additional details, refer to the docs.
• Introduced configuration validation to avoid setting the garbage collection threshold below the max task runtime, preventing premature Pod removal and task interruptions.
• Modified the task agent directory to avoid conflicts with existing directories in the PATH variable.

Both Machine and Container Runner:

• Added injection of the CIRCLE_URL variable into task environments. This will resolve to https://runner.circleci.com on cloud, or to the host URL of your server installation on enterprise server.

For customers trying to find cost efficiencies in their runner environment we now support spot instances. The feature can also improve reliability when there are unexpected failures. It comes with some risks and we have documented risks and usage. This release also includes some enhancements and security fixes.

Container Runner:

• Support for unsafe retries, which enable container-agent to automatically rerun tasks that get interrupted during their execution. For example, unsafe retries can be used to run idempotent workloads on spot instances for cost-saving benefits without the worry of Pod preemption causing unexpected job failures. For important information and risk assessment on setting up and using unsafe retries, please read the docs: https://circleci.com/docs/container-runner/#unsafe-retries
• Changed the format of the arbitrary suffix appended to the names of Kubernetes objects that are managed by container-agent. This change has no operational impact on runner.

Both Machine and Container Runner:

• The CircleCI CLI is now prioritized over task agent on the $PATH. That is, invoking the circleci command within a task will point to the CLI binary, if present.
• Patched with Go v1.21.10 for security fixes in Go: https://groups.google.com/g/golang-announce/c/wkkO4P9stm0

Note: If you are running CircleCI Server, you will need version 4.4.3 of server to take advantage of these runner features.

Container Runner

• Reduce long term memory footprint of Container Runner
• Retry tasks where the pod has failed to start due to pod startup errors
• Improve Container Runner monitoring mechanisms for active task pods
• Use the home SSH directory for checkout keys instead of a temporary directory

Machine Runner

• Optionally use the user’s home SSH directory for checkout keys instead of a temporary directory

Critical security fix

• Patched with Go v1.21.5 for critical security fixes in Go. Go release announcement: https://groups.google.com/g/golang-announce/c/iLGK3x6yuNo

Enhancements to error logging and event handling:

• Now logging the outcome of the task Pod’s PreStop hook

• Clearer and actionable error messaging when the task Pod fails to startup or its liveness probe fails

General improvements

• The garbage collector period is now configurable for tuning how quickly failed Pods are removed

• Ability to customize individual values (e.g., the timeoutSeconds) on the task Pod’s primary container’s liveness probe. Any overrides are merged with the defaults, like a kubectl patch.

• See runner configuration documentation for all configuration options

Container Runner SSH Rerun (Preview)

Container runner now supports SSH Rerun. Visit our documentation on setup and usage.

For more information on SSH rerun please see the following documentation.

Machine runner 3.0 is now Generally Available, providing improvements to usability, setup, and maintenance of machine runners.

Container runners and machine runners are now built using the same framework. Launch agent users will need to upgrade to the new application, but existing container runner installations will not require any additional action.

Faster feature enhancements: We’ll now bring many network resilience feature enhancements to both types of self-hosted runners (machine and container).

Improved usability: With more uniform logs and metrics, CircleCI’s machine runners ensure consistent and reliable performance and a seamless experience for users regardless of their runner setup.

Setup and maintenance: CircleCI’s machine runner 3.0 simplifies the setup and maintenance process by offering convenient packages that allow you to easily integrate with your preferred cloud templating tools.

Linux, macOS, and Windows support is available. See all the details on the runner documentation.

Machine Runner 1.1 variants will be supported through March of 2024, giving you time to plan a migration.

View our documentation to upgrade.