We’ve added additional meta data using the “tag” key to audit logs to help customers identify how workflow reruns were triggered.

Workflow tags include:

• setup
• rerun-single-job
• rerun-workflow
• rerun-workflow-from-beginning
• rerun-workflow-from-failed
• rerun-workflow-from-job-ids-sparse-tree
• rerun-single-job-with-ssh

Example audit log entries

Workflow Started:

workflow.start
{"job":{},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

workflow.job.context.request
{"job":{"context_uuids":[],"contexts":[],"id":"408...","job_name":"test","job_status":"pending"},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

workflow.job.start
{"job":{"contexts":[],"id":"408...","job_name":"test","job_status":"running"},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

workflow.job.finish
{"job":{"contexts":[],"id":"408...","job_name":"test","job_status":"failed"},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

Retry the original workflow with rerun job with SSH enabled:

workflow.retry
{"job":{},
 "workflow":{"id":"247...","name":"test-and-build","tag":null}}

SSH enabled rerun started:

workflow.start
{"job":{},
"workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

workflow.job.context.request
{"job":{"context_uuids":[],"contexts":[],"id":"c6a...","job_name":"test","job_status":"pending"},
 "workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

workflow.job.start
{"job":{"contexts":[],"id":"c6a...","job_name":"test","job_status":"running"},
 "workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

workflow.job.finish
{"job":{"contexts":[],"id":"c6a...","job_name":"test","job_status":"canceled"},
 "workflow":{"id":"811...","name":"test-and-build","tag":"rerun-single-job-with-ssh"}}

The pipelines page can load slowly for some users. The number of pipelines and number of projects can contribute to that slowness. A recent change to one of the underlying data services has been made resulting in a 30%-40% improvement in performance.

We continue to work on improving the pipelines page experience. Our current focus is on load time. We do have plans to introduce new filter and search capabilities in the future.

We have added an API and UI component to approval jobs that allows you to “cancel” the approval job. Previously it was not possible to cancel approval jobs. This limitation forced you to either approve the job or let it expire, which takes 90 days. This led to a suboptimal experience with jobs left in a non-terminal state. This could impact performance metrics, as well pipelines views. It also led to non-intuitive actions being taken like cancelling the workflow which cancels all remaining jobs.

We are following up this feature with more flexible require statements. So you can have a job depend on an approval while another depends on cancellation. This is great for cleanup jobs, notifications, or other tasks that may only need to happen or not happen upon cancellation.

For CircleCI organizations that integrate with GitLab or GitHub App:

• The user count on the People tile reflects the number of users in the organization instead of active users in the billing period.

• Organization admins have the ability to invite additional teammates directly from the Organization Homepage by clicking the “Invite your teammates” button.

The CircleCI Automation Actor will be used to attribute automated workflow and job cancelations on September 17th, 2024. This change will impact two functionalities:

• Auto-cancel redundant workflows
• Jobs canceled via the v1 Cancel Job API using a project token

This change is intended to better indicate that the action was performed by a machine.

Once the change is completed, the canceled_by field of the getWorkflowById and the listWorkflowJobs API response will return the CircleCI Automation Actor ID. You only need to take action if you rely on this field to have a static value.

Previously: The pipeline.trigger_parameters.webhook.body pipeline value was only populated when a custom webhook triggered the pipeline. If a config used pipeline.trigger_parameters.webhook.body and the pipeline was triggered with a GitHub App event, an errored pipeline would occur. This made it difficult to use one configuration file with a custom webhook & GitHub App triggers.

New behavior: The pipeline.trigger_parameters.webhook.body pipeline value will be populated with an empty string if the pipeline is triggered from a GitHub App event. This simplifies using one configuration file with a custom webhook & GitHub App triggers.

Three new fields will be introduced to the Usage API on August 23, 2024 to help you better understand how your builds are running. These fields are:

• job_run_queued_at: the timestamp (UTC) of when the job started to queue (if applicable)
• job_run_started_at: the timestamp (UTC) of when the job started to run
• job_run_stopped_at: the timestamp (UTC) of when the job stopped (only logged for success and failed job build statuses)

These three new fields will be introduced into the CSV output in the order above immediately following the job_run_date and prior to the job_build_status fields.

NOTE: If you have an automated ingestion process that relies on the format of the CSV remaining consistent, please update your ingestion process to accommodate these three new fields prior to August 23, 2024.

Admins must now opt in their organization to use the experimental intelligent error summaries. Enable this experiment in the Advanced section of Organization Settings.

The feature is currently powered by OpenAI. The only data sent to OpenAI is the error message in the output from the failed step. Before the message is sent any sensitive information such as passwords or access keys is removed. None of the data provided is being used to train the model.

Let us know what you think in the Discuss thread.

NOTE: Release 3.0.23 has been removed from our package repository. It contained bug breaking OIDC token injection for Cloud Runner users. Please use this release 3.0.24 instead.

Container Runner

• Added support for customizable service containers, allowing users more control over a service’s resource usage on a per-image basis. For additional details, refer to the docs.
• Introduced configuration validation to avoid setting the garbage collection threshold below the max task runtime, preventing premature Pod removal and task interruptions.
• Modified the task agent directory to avoid conflicts with existing directories in the PATH variable.

Both Machine and Container Runner:

• Added injection of the CIRCLE_URL variable into task environments. This will resolve to https://runner.circleci.com on cloud, or to the host URL of your server installation on enterprise server.

There have been several updates to the CircleCI web app

• The term “Dashboard” has been replaced by “Pipelines” in the left-hand navigation
• There is a new “Organization Home” option that will show an overview of the organization that you have currently selected including the projects in that organization that you have followed
• Upon clicking on an org from the “User Homepage”, you will be taken to the “Organization Home” instead of the “All Pipelines” page
• When creating a new organization on CircleCI, there is no longer a question asking to connect a VCS

For customers trying to find cost efficiencies in their runner environment we now support spot instances. The feature can also improve reliability when there are unexpected failures. It comes with some risks and we have documented risks and usage. This release also includes some enhancements and security fixes.

Container Runner:

• Support for unsafe retries, which enable container-agent to automatically rerun tasks that get interrupted during their execution. For example, unsafe retries can be used to run idempotent workloads on spot instances for cost-saving benefits without the worry of Pod preemption causing unexpected job failures. For important information and risk assessment on setting up and using unsafe retries, please read the docs: https://circleci.com/docs/container-runner/#unsafe-retries
• Changed the format of the arbitrary suffix appended to the names of Kubernetes objects that are managed by container-agent. This change has no operational impact on runner.

Both Machine and Container Runner:

• The CircleCI CLI is now prioritized over task agent on the $PATH. That is, invoking the circleci command within a task will point to the CLI binary, if present.
• Patched with Go v1.21.10 for security fixes in Go: https://groups.google.com/g/golang-announce/c/wkkO4P9stm0

Before upgrading

See the CircleCI server 4.5 release notes and upgrade guide for this release.

Vault is being deprecated and will no longer be supported in server 5.0. Refer to our script for steps to migrate to Tink.

What’s new in Release 4.5.0

The v4.5 release introduces security and performance improvements.

New features

• You can now restrict access to contexts using expression restrictions. Read more in the docs.
• You can now configure the Nomad Autoscaler evaluation_interval and cooldown.

Changes

• Very large and complex configuration files, notably those using a high number of pipeline parameters, will now have a shorter compile time.
• Optimizations have been made in orb-service for a reduction in both storage space used and timeouts for very large configuration compilation.
• The login page has been replaced with a new one. When logging in to server now you will be redirected to /server-login. The banner from the login page has been removed.
• Selecting items in the navigation bars of the settings pages no longer triggers a full page reload.
• Project and personal access tokens are now hashed rather than being stored as plaintext.
• Obsolete caches have been removed leading to improve Workflows Conductor performance and reduced Redis usage.
• Required domain-service caching variables are now defined by default.
• Various changes have been made in the UI to standardize style and icon usage.

Bug fixes

• Fixed a bug that was causing duplicate workflows showing in the UI.
• Fixed a bug that was causing config compliation checks to attempt to access an invalid subdomain.
• The self-hosted runners Create Resource Class button in the web app has been fixed.

New services

New services introduced with this release:

• ciam-gateway
• ciam
• web-ui-authentication

Database migrations

The following databases will run migrations when upgrading to this version:

• builds_service
• permissions
• runneradmin
• conductor_production

Known issues

• Vault may not refresh its client token after a month of uptime. Migrate to Tink to resolve this issue.
• Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for public installations, but for private installs you would need to ensure that you can access the private IP advertised. For example, by using a VPN into your VPC.
• CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.
• Customers installing Server to GKE should wait to upgrade until further notice. There is an issue preventing a successful upgrade. New installations are not impacted.

To learn more about Server 4.5 installation, migration, or operations please review our documentation.

Projects integrated through GitHub App now support Auto-Cancel Redundant Workflows.

This feature helps reduce spend and queuing for highly active projects. It can be enabled by navigating to Projects Settings > Advanced > Auto-cancel redundant workflows.

For more information, see our documentation.

We are introducing parameter limits to improve responsiveness of the trigger pipeline API and limit potential abuse. The new limits will be as follows:

• 100 max entries
• 128 maximum key length
• 512 maximum value length

For those customer projects that have historically gone over these limits we have made project level exceptions. We are evaluating those projects to determine alternative methods to address their needs.

Note: If you are running CircleCI Server, you will need version 4.4.3 of server to take advantage of these runner features.

Container Runner

• Reduce long term memory footprint of Container Runner
• Retry tasks where the pod has failed to start due to pod startup errors
• Improve Container Runner monitoring mechanisms for active task pods
• Use the home SSH directory for checkout keys instead of a temporary directory

Machine Runner

• Optionally use the user’s home SSH directory for checkout keys instead of a temporary directory

An “unauthorized” error message is shown when a user does not have access to contexts required to run the job.