Releases overview

Cloud

Introduction

CircleCI releases gives you visibility into your Kubernetes deployments within the CircleCI web app, as well as access to commands and visualisations for release management. From the CircleCI web app, you can:

Restore version
Scale component
Restart component

We support Argo Rollouts for progressive delivery. Currently the canary deployment strategy is supported, but in a future release blue-green will also be supported. Some further controls are available for projects that use Argo Rollouts:

Retry release
Promote release
Cancel release

Quickstart

If you want to get started with CircleCI releases for Kubernetes right away, you can jump to:

To get set up with releases for Amazon SageMaker, refer to the Deploy to Amazon SageMaker page.

How it works

CircleCI integrates with your Kubernetes cluster to give you visibility and control over your release process through the CircleCI web app. We currently support Kubernetes Deployments and Argo Rollouts.

Diagram showing the architecture of CircleCI releases

Configure environment integrations to install the CircleCI release agent into your Kubernetes clusters. Configure the release agent to only monitor the namespaces you require.

Developers can monitor releases in the CircleCI web app, and run commands to manage those releases from the UI. Commands are monitored by CircleCI and relayed to the release agent, which then runs the commands for you.

Concepts

The sections below explain some key software delivery concepts. Understanding these concepts will help you take full advantage of CircleCI’s release features.

Component

A component in CircleCI is a collection of code and configuration that is deployed and released as a single unit. In Kubernetes terms, this would be a Deployment or Rollout object along with the related objects such as Pods, ReplicaSets, etc. that share a common circleci.com/component-name label.

Delivery

Delivery is the act of packaging code changes and making them available for Deployment. Continuous delivery is the prerequisite step for continuous deployment. With some variations on the technologies being used, the delivery process creates executables from code and then makes them available from a repository to be deployed to an environment at a subsequent time.

Deployment

Deployment is the act of putting a new component version into an environment, regardless of whether users and other services interact with the new version or a previous one. Depending on the deployment type, a release may happen as a later task (such as switching over traffic shaping rules for a blue/green deployment) or as a direct consequence of the deployment (such as a standard Kubernetes rolling update).

Command

A command is a user-initiated action CircleCI performs on the user’s behalf to manipulate a specific component. These actions are run asynchronously via our release agent and the results are reported back to the CircleCI web app where you can see them in the releases dashboard, similar to how step output works for CI jobs.

Some commands are available for all components. These are:

Restore version
Scale component
Restart component

A subset of commands are available for progressive releases (when using Argo Rollouts). These are:

Retry release
Promote release
Cancel release

Release

A release is the act of updating a component to a new version in a specific environment, causing it to become available to an audience of users and other services.

A release can happen when an existing component is updated or when the first version for a new component is deployed.

In the releases dashboard, releases are tagged as Progressive if they refer to an Argo Rollout.

A release starts when the component version is updated, and ends when the new version has reached 100% availability, and all associated validations have been completed. In the case of a progressive release, this happens when the Rollout completes. In the case of a Deployment, this happens when the Deployment replicas are all available and ready.

The releases UI

The CircleCI releases UI is a powerful tool for visualising, monitoring and managing your releases. The pages included in the releases UI are described below.

The dashboard

Screenshot showing the releases dashboard in the CircleCI web app

Select Releases in the CircleCI web app sidebar to enter the releases dashboard. The dashboard shows the following:

Timeline: A timeline of releases across your organization’s components and environments. You can use the filter drop downs at the top to select a component and/or environment. Information you see here includes live status updates, release trigger source, release version, release type (for example, progressive, when you are using an Argo Rollout). From here you can access:
- The release details page for a specific release by clicking on the status badge or version number.
- The component or environment details pages by clicking on the respective names.
- The project dashboard for the CircleCI project associated with a release.
- The job details page in the CircleCI web app for the deployment job that kicked off the release.
- The commit details page in your VCS for the commit that started the deployment.
Environments: List of environment integrations set up for your organization. From here you can:
- Set up a new environment integration by selecting Create Environment Integration.
- Access settings for each environment ().
- Access the environment details view by selecting an environment name.
Components: A list of components and their associated projects. From here you can:
- Set up a new component by selecting Create Component. From here you can select and environment that has a successful environment integration set up, and from there you can add a new component.
- Get straight to the component’s project building on CircleCI by selecting the project name.
- Access setting for each component ().

Filter and group components and environments with labels

Add labels to your components and environments to provide teams with a way to filter and group content in the releases UI. Once a label is added, you can use this to filter your view to focus on the content relevant to your team.

Labels are composed of two values separated by a colon, for example, team:my-team-name. To specify multiple labels for a component or environment, you can separate them with a comma. For example, team:my-team-name, role:web. You can add up to 20 labels to a component or environment.

Use labels to filter components and environments

Once you have added labels to your components and environments, you can use them to filter your view in the releases UI. In the timeline, environments, or components view, select a filter to reduce the content in the tab to only your selection. You can also use the label filter dropdown menu at the top of the page.

Add or edit labels

To add or edit labels follow the steps below.

Screenshot showing the location of the add/edit labels button

Figure 1. Add and Edit environment and component labels

Component labels

To add or edit labels for a component, follow these steps:

Select Releases in the CircleCI web app sidebar.
You are now in the timeline view. Select the Components tab.
Select the cog icon () for your component. You can use the filter at the top of the page to help find the component you want.
You are now on the component settings page. Select the edit button () in the labels panel.
Enter or edit your label(s) and select Done.

Environment labels

To add or edit labels for an environment, follow these steps:

Select Releases in the CircleCI web app sidebar.
You are now in the timeline view. Select the Environments tab.
Select the cog icon () for your environment.
You are now on the environment settings page. Select the edit button () in the labels panel.
Enter or edit your label(s) and select Done.

View all releases for an environment

To view all releases for an environment, follow these steps:

Select Releases in the CircleCI web app sidebar.
You are now in the timeline view. Select the Environments tab.
Select your environment by name.
You are now on the environment details page. Select the Releases tab to view a list of all releases for your chosen environment.

View all commands run for an environment

To view all commands run for an environment, follow these steps:

Select Releases in the CircleCI web app sidebar.
You are now in the timeline view. Select the Environments tab.
Select your environment by name.
You are now on the environment details page. Select the Commands tab to view a list of all commands that have been run for your chosen environment.

View all releases for a component

To view all releases for a component, follow these steps:

Select Releases in the CircleCI web app sidebar.
You are now in the timeline view. Select the Components tab.
Select your component by name. You can use the filter at the top of the page to help.
You are now on the component details page. Select the Releases tab to view a list of all releases for your chosen component.

View all commands run for a component

To view all commands run for a component, follow these steps:

Select Releases in the CircleCI web app sidebar.
You are now in the timeline view. Select the Components tab.
Select your component by name. You can use the filter at the top of the page to help.
You are now on the component details page. Select the Commands tab to view a list of all commands run for your chosen component.

Requirements

Tooling

We test the versions listed here. Older versions may work but are not guaranteed.

Tool	Tested versions
Kubernetes	1.29.0 1.28.0 1.27.0
Helm	3.12.0 3.11.0
Argo Rollouts (optional)	1.6.0 1.5.0

Labels and annotations

The following table shows a complete list of labels and annotations either required or available for configuring your environment integration.

	Label/annotation	Value	Required?
`Metadata.Labels`	`circleci.com/component-name`	A name for your application	Yes
`Metadata.Labels`	`circleci.com/version`	Current version	Yes
`Spec.Template.Metadata.Labels`	`circleci.com/component-name`	See above	Yes
`Spec.Template.Metadata.Labels`	`circleci.com/version`	See above	Yes
`Metadata.Annotations`	`circleci.com/project-id`	Project ID for the CircleCI project associated with the job that deploys your component	Yes
`Metadata.Annotations`	`circleci.com/operation-timeout`	A Go duration string, for example, 5m, 10m15s	No. Only needed to set a custom timeout duration. This option is only available if you are using Helm to configure your Kubernetes resources.
`Metadata.Annotations`	`circleci.com/restore-version-enabled`	`false`	No. Only set if you want to disable the restore version feature for your component.
`Metadata.Annotations`	`circleci.com/scale-component-enabled`	`false`	No. Only set if you want to disable the scale component feature for your component.
`Metadata.Annotations`	`circleci.com/restart-component-enabled`	`false`	No. Only set if you want to disable the restart component feature for your component.
`Metadata.Annotations`	`circleci.com/retry-release-enabled`	`false`	No. Only set if you want to disable the retry release feature for your component.
`Metadata.Annotations`	`circleci.com/promote-release-enabled`	`false`	No. Only set if you want to disable the promote release feature for your component.
`Metadata.Annotations`	`circleci.com/cancel-release-enabled`	`false`	No. Only set if you want to disable the cancel release feature for your component.

Autodetected releases and release markers

CircleCI automatically generates release markers for your deployment pipelines. Release markers provide a lightweight way to log your releases without requiring a full CircleCI releases setup. Release markers create for you a log of all releases in one place, for a clear overview of what has changed, without the need to search through your CI/CD pipelines. Release markers log all new releases in one place and link back to the CI/CD pipelines that caused them. You can use release markers independently, without installing the release agent.

Autodetection of releases

CircleCI looks for the word deploy in your job names to detect when a release may have happened and creates an autodetected release marker.

When CircleCI creates a release marker for you automatically, we use the information available in your pipeline configuration to create the following:

Component: CircelCI created a component for the release and uses your project name to create a component name.
Environment integration: CircleCI creates an enviornment for you, using your job and workflow names to attempt to create a relevant name. If an environment name can not be generated, the name will be autogenerated, and you can edit it later. The environment has the type AUTODETECTED.
Release version: CircleCI uses the build number to populate the release version for the purposes of displaying the release in the CircleCI web app.

If any of the following conditions are true, CircleCI will not generate a release marker for a job even if the name contains the word deploy:

The job has a status other than SUCCESS.
Any of the following keywords and keyword combinations are present in the job name:
- dry run
- skip deploy
- bypass deploy
- deployed
- validation
- validate
The project related to the job has releases created via the Kubernetes release agent or manual release jobs in the last two months.
You have opted out of the autodetected releases feature.
The detected environment integration name collides with an integration you have already set up.

Manual configuration of autodetected releases

Some points to consider when manually configuring autodetected releases:

If you decide you would like to manually configure a release marker for an environment integration that was created as part of an autodetected release, refer to the Configure release markers page to set up a release marker and a new environment integration.
If you change a component name that was originaly autodetected by us, CircleCI creates a new component and the history will not carry over. If you do not update the component name, the history will carry over.
If your environment integration has an autogenerated name, you can edit the name as follows:
1. In the CircleCI web app, select your organization.
2. Select Releases in the sidebar.
3. Select the Environments tab.
4. Find your environment and select cog icon () to enter the environment integration settings page.
5. Select Edit.
6. Enter a new name and select Update.

Opt out of autodetected releases

If you do not want CircleCI to automatically create release marker for you, you can opt out of this feature either for a whole organization or per-project.

To opt out of autodetected releases for an entire organization, follow these steps:

In the CircleCI web app, select your organization.
Select Organization settings in the sidebar.
Select Releases.
Toggle the "Enable automatic release marker detection" option left. You will see a confirmation message in the bottom right hand corner of the app. To re-enable the feature, toggle the option right.

To opt out of autodetected releases for a project, follow these steps:

In the CircleCI web app, select your organization.
Select Projects in the sidebar.
Find your project in the list, select the ellipsis (), select Project Settings.
Select Releases.
Toggle the "Enable automatic release marker detection" option left. You will see a confirmation message in the bottom right hand corner of the app. To re-enable the feature, toggle the option right.

Manual setup of release markers

If you would rather set up release markers yourself, see the Configure release markers page.

Release status

A release can be in one of the following states:

Status	Notes
RUNNING	The release is currently in progress.
FAILED	Resources have reached an unhealthy status (pods for the new version of a Kubernetes component).
SUCCESS	The Deployment or Rollout has all desired resources available (all pods specified by a Kubernetes Deployment or Argo Rollout).
CANCELLED	The release has been cancelled, either using the `cancel release` option, or by being superseded by another release.
EXPIRED	Release commands failed to be picked up by the release agent within the required time window.
LOGGED	Release has been logged using a release marker and is available in the CircleCI releases UI.

Known limitations

Restarting the release agent while a release is ongoing will cause the release agent to lose track of the release status and fail to update the CircleCI services accordingly.
In the CircleCI releases UI it is currently possible for you to attempt and restore a version that does not exist. All releases are presented in the UI, including those outside of the scope of any version history limits you might have set. We do not currently filter out releases for which there is no longer any data.

Depending on your setup, you will have options for configuring revision history limits: revisionHistoryLimit for Kubernetes and Argo Rollouts, and $HELM_MAX_HISTORY for Helm.

If you have these limits set, you can not restore a version outside the limit. For example, if your limit is set to the last 10 releases, you can not restore the 11th deployment back.

We are working on updates to:
- Indicate out-of-scope releases
- Prevent you from attempting to restore unavailable releases
- Provide a manual way for you to mark releases as unavailable

Troubleshooting

Why is my Deployment/Rollout not showing up in the components tab or releases timeline view?

Check that the Deployment/Rollout is annotated with the required labels. More information is available in the Set up guides. If the required labels were not present, then adding them should solve the problem.
If you are using a Deployment, check that the desired replicas is not set to 0. Deployments with 0 replicas are not reported as releases, even if they are scaled up subsequently. The configured value can be seen on the release agent deployment in the circleci-release-agent-system namespace. Here is an example in which the number of desired replicas is 2:
```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-deployment
  namespace: sample-namespace
spec:
  replicas: 2
```
Check that the Deployment/Rollout is in a namespace managed by the release agent. This can be verified by checking the MANAGED_NAMESPACES environment variable on the release agent deployment in the circleci-release-agent-system namespace. Here is an example in which only the default namespace is being managed:
```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: circleci-release-agent
  namespace: circleci-release-agent-system
spec:
  template:
    spec:
      containers:
      - env:
        - name: MANAGED_NAMESPACES
          value: default
```

Why is my release is stuck in the `Running` state?

If you are using a Deployment, check whether it was deleted before the release could complete. In this scenario, this is an expected behavior. This experience will be improved upon in future release agent updates.
If you are using a Deployment, check whether the release agent restarted before all pods for the deployment could become ready. This is a known limitation that will be addressed in future updates of the release agent. Restarting a release agent while a release is ongoing will cause the release agent to lose track of the release status and fail to update the CircleCI services accordingly.

Why are no new releases are showing up? and/or why are component versions not being updated?

Check whether the token used by the release agent has been revoked:
1. Select Releases in the CircleCI web app sidebar
2. Select Configure Environments to enter the release environments view
3. Select your environment to view valid token details, including when the token was last used.
  
  If the token has been last used longer than a minute ago, then this is likely to be the problem.
Check whether tokens are being shared between multiple release environments. This is not supported. Check this by following these steps:
1. Retrieve the token value from the token field in the circleci-release-agent secret in the circleci-release-agent-system namespace
2. Compare the value with the partially obscured value for the available Tokens in the CircleCI web app
  
  If the token does not show up in the list, then it has been revoked or the value configured on the release agent is incorrect. In either case, creating a new token and reinstalling the Release Agent with the new value should solve the issue.

Why is `restore version` using Helm is timing out?

The time required for a Helm-based restore version to complete successfully is dependent on the specific configuration of the target component. For example, a large number of replicas will lead to a longer duration, which could cause a timeout. It is possible to specify a different timeout by adding the circleci.com/operation-timeout annotation to the Rollout or Deployment. The default value for this is 10 minutes. For steps see the Configure your Kubernetes components page.

Why is the restore version button not available for a component version?

Check whether the component has been undeployed. If there are currently no live versions for a component, the Restore Version button will not be visible for that component until at least one version has been deployed.

Why are all buttons disabled for a release?

Check whether the release is a Rollback. If this is the case, then this is a known issue that will be solved in a future update to the CircleCI release agent.

Why are all commands for my component failing?

Check if the error message is “invalid or missing project ID“. In this case the component is missing a valid Project ID.

Next steps

Set up CircleCI releases
To try out setting up CircleCI releases for a Kubernetes deployment, visit the Continuous delivery config examples repo.

Suggest an edit to this page

Make a contribution

Learn how to contribute

Still need help?

Ask the CircleCI community

Join the research community

Visit our Support site