View entire changelog

Release 3.1.0

Before Upgrading

IMPORTANT: With this release, the frontend-external load balancer has been removed. The traefik load balancer now handles all incoming traffic. When updating from a previous server 3.x version, you will need to update the DNS record that was pointing to the frontend-external load balancer and have it point to the circleci-server-traefik load balancer instead. Remember, you can retrieve the external IP address or DNS name of your traefik load balancer by typing kubectl get svc/circleci-server-traefik in a terminal that has access to the cluster.

For further information see the What’s New doc.

What’s New in Release 3.1.0

New Features

  • Telegraf plugins can now be added to server and customized to use third party monitoring solutions, for example, Datadog. For more information, see the Metrics and Monitoring doc.
  • The option to use only private load balancers has been introduced for customers who want a fully private installation. For more information see the Load Balancers guide.
  • Server 3.x hosts build artifacts, test results, and other state in object storage. We support any S3-compatible storage and Google Cloud Storage. For more information, see the Installation guide for further information.
  • Dynamic config via setup workflows is now available on server installations. For more information see our blog post and the Dynamic Configuration docs page.
  • Runner is now available on server. For further information, including installation steps, see the Runner docs. Runner allows the use of the macOS executor in server installations and VM service functionality for customers with server installed in a private data centre.
  • The frontend load balancer from v3.0 has been removed and replaced with an Ingress resource and the Traefik Ingress controller. This is a breaking change requiring you to reconfigure your DNS. See the What’s New in server docs for further information and guidance.
  • The following services can now be externalized. For setup information, see the server v3.x installation guide:
    • Postgres
    • MongoDB
    • Vault
  • Backup and restore functionality is now available. For more information see the Backup and Restore guide.
  • Prometheus is now deployed by default with server to monitor your cluster health and usage. Prometheus can be managed and configured from the KOTS admin UI. For further information, see the Metrics and Monitoring doc.
  • Server now supports the 2XL resource class. The Nomad cluster needs to be made large enough to account for larger resource classes.
  • The lifecycle of build artifacts and test results can now be configured from the KOTS admin console under Storage Object Expiry, including the option to disable the expiration and retain artifacts and test results indefinitely.


  • Resolved a collection of bugs that were causing sensitive information to be leaked into CircleCI support bundles:
    • Instances of faulty and partial redactions of secrets were detected, in part due to 3rd party bugs.
    • PostgresDB leaking sensitive information to STDOUT.
    • Several CircleCI services were logging secrets.
  • Tightened network security in the Nomad terraform module.
  • Terraform v0.15.0 and up are now supported.
  • Updated installation scripts to use functions supported by most recent versions of Terraform.
  • Resolved a bug that was leading to machine large builds being run on the wrong machine type. Machine large builds now correctly use 4 vCPUs and 16GB of RAM.
  • Resolved a bug that caused contexts-service to fail on expiration of Vault client tokens.
  • Resolved a bug that was causing legacy-notifier to report readiness prematurely.
  • The JVM heap size parameter has been removed for all services. The heap size is set to be half of the memory limit.
  • Changes to networking config and certs are now picked up automatically by Traefik. Previously, a restart would have been required.
  • Minimum requirements for CPU and memory have changed. For the new values, see the Installation Prerequisites doc.

Known Issues

  • Retry with SSH for jobs using the machine executor advertises a private IP address. For this reason, retry with SSH for jobs using the machine executor works as standard for private installations, but for public installs you would need to ensure that you can access the private IP advertised, for example, by using a VPN into your VPC.
  • It is currently possible for multiple organizations under the same CircleCI server account to have contexts with identical names. This should be avoided as doing so could lead to errors and unexpected behavior.
  • CircleCI 1.0 builds are not supported. If an attempt is made to run a 1.0 build, no feedback will be available in the application to indicate the cause of the issue. If a build is run on your installation and does not show up in the CircleCI application, users should be directed to use the CircleCI CLI to validate the project configuration and get details of the possible cause of the issue.

To learn more about Server 3.0 installation, migration, or operations please see our documentation.

Previous changes

Xcode Updates