Phase 2 - Core services
On This Page
Before you begin with the CircleCI server v4.x core services installation phase, ensure all prerequisites are met.
| In the following sections, replace any sections indicated by |
1. Create a namespace
Create a namespace to install the application into.
kubectl create ns <namespace>
| Once you have created your namespace, we recommend setting your |
2. Pull images
Credentials to pull the images from CircleCI’s image registry will be provided to you as part of the onboarding process. A
docker-registry Kubernetes Secret will be used to pull images from Azure Container Registry (ACR). You have two options, depending on whether your application has access to the public internet.
3. Create helm values
Before installing CircleCI, it is recommended to create a new
values.yaml file unique to your installation. The Installation Reference section contains some example
values.yaml files that are a good place to start. The following describes the minimum required values to include in
values.yaml. Additional customizations are available, see the provided
values.yaml for all available options.
For sensitive data there are two options:
add into the
add them as Kubernetes Secrets directly
This flexibility allows you to manage Kubernetes Secrets using whichever process you prefer. Whichever option you choose, this sensitive information is stored as a Kubernetes Secret within CircleCI.
| During the installation process, you may use the following command to generate a random alphanumeric value as required: |
a. API token
The application requires a Kubernetes Secret containing an API token. This API token is used to facilitate internal API communication to api-service. Use a random string and store it securely, CircleCI will not be able to recover this value if lost. There are two options depending on whether you want to create the Kubernetes Secret, or if you want CircleCI to create it for you.
b. Session cookie
The application requires a session cookie key Kubernetes Secret, which CircleCI uses to sign session cookies. The Secret must be exactly 16 characters long. Use a random string and store it securely, CircleCI will not be able to recover this value if lost. There are two options depending on whether you want to create the Kubernetes Secret, or if you want CircleCI to create it for you.
The application requires a Kubernetes Secret containing signing and encrpytion keysets. These keysets are used to encrypt and sign artifacts generated by CircleCI. These keys were created during the prerequisites phase. CircleCI will not be able to recover the values if lost. Depending on how you prefer to manage Kubernetes Secrets, there are two options.
The application requires a Kubernetes Secret containing Postgres credentials. This is true when using either the internal (default) or an externally hosted instance of Postgres. CircleCI will not be able to recover the values if lost. Based on how you prefer to manage Kubernetes Secrets there are two options.
Postgres may be extended to use TLS encrpted traffic. When deployed internally, this option is disabled by default but may be enabled by adding the following to your postgresql block of your
postgresql: ... tls: enabled: true autoGenerated: true # Generate automatically self-signed TLS certificates
Certificate files may also be provided, rather than autogenerated. In this case, create a secret containing the TLS certs and keys needed.
kubectl create secret generic postgres-tls-secret --from-file=./cert.pem --from-file=./cert.key --from-file=./ca.pem
Then the postgresql block in your
values.yaml will contain the contents below.
postgresql: ... tls: enabled: true certificatesSecret: "postgres-tls-secret" # Name of an existing secret that contains the certificates certFilename: "cert.pem" # Certificate filename certKeyFilename: "cert.key" # Certificate key filename certCAFilename: "ca.pem" # CA Certificate filename
e. MongoDB credentials
The application requires a Kubernetes Secret containing MongoDB credentials. This is true when using either the internal (default) or an externally hosted instance of MongoDB. CircleCI will not be able to recover the values if lost. Based on how you prefer to manage Kubernetes Secrets there are two options.
f. RabbitMQ configurations and auth Secrets
The RabbitMQ installation requires two random alphanumeric strings. CircleCI will not be able to recover the values if lost. Based on how you prefer to manage Kubernetes Secrets there are two options.
g. Pusher Kubernetes Secret
The application requires a Kubernetes Secret for Pusher. CircleCI will not be able to recover the values if lost. Based on how you prefer to manage Kubernetes Secrets there are 2 options:
All values in this section are children of
global in your
CircleCI domain name (required)
Enter the domain name you specified when creating your Frontend TLS key and certificate.
global: ... domainName: "<full-domain-name-of-your-install>"
A license will be provided by CircleCI, add it to
global: ... license: '<license>'
The registry to pull images from will have been provided to you, or you may have added the images to your own hosted registry. Add the registry to
global: ... container: registry: <registry-domain eg: cciserver.azurecr.io > org: <your-org-if-applicable>
i. Static IPs
If you provisioned a Static IP (GCP) or Elastic IPs (AWS) in the prerequisites, you can now add the values under the nginx block.
GCP - Add Static IP
For GCP, add the provisioned IPv4 address under the
loadBalancerIp field in the nginx block.
nginx: ... loadBalancerIp: "<gcp-provisioned-ipv4-address>"
AWS - Add Elastic IPs
For AWS, under nginx annotations, add the
service.beta.kubernetes.io/aws-load-balancer-eip-allocations annotation with each of the
AllocationId values generated as a comma separated list. Please note, the number of `AllocationId`s must match the number of subnets the load balancer is deployed into (default 3).
nginx: ... annotations: ... service.beta.kubernetes.io/aws-load-balancer-eip-allocations: <eip-id-1>,<eip-id-2>,<eip-id-3>
For TLS, you have 4 options:
k. GitHub integration
To configure GitHub with CircleCI, there are two options for providing credentials to the deployment. Steps for both GitHub and GitHub Enterprise (GHE) are given in the next two sections.
These instructions are for the GitHub.com, not GitHub Enterprise. Use the client ID and secret you created with your Github OAuth application in the prerequisites phase.
The instructions for GitHub Enterprise are similar, with a few extra steps to enable Enterprise and create the required default token.
In the case of GitHub Enterprise add the
defaultToken created in the prerequisite phase to the
GitHub section. The hostname should not include the protocol, ex:
l. Object storage
Regardless of your storage provider, the bucket name you created during the prerequisites phase will need to be included.
object_storage: bucketName: "<bucket-name>"
s3 section as a child of
endpoint in the case of AWS S3 is the regional endpoint, it is of the form
https://s3.<region>.amazonaws.com. Otherwise it is the API endpoint fo your object storage server.
object_storage: ... s3: enabled: true endpoint: "<storage-server-or-s3-endpoint>"
object_storage.s3, you may provide the
irsaRole, or nothing. They were created during the prerequisites steps.
CircleCI server will use the role provided to authenticate to S3.
Google Cloud Storage
object_storage add the following.
gcs: enabled: true
object_storage.gcs you may add
workloadIdentity, or neither. The keys/role were created during the prerequisites steps.
m. Installing behind a proxy
Depending on your security requirements, you might want to install CircleCI server behind a proxy. Installing behind a proxy gives you the power to monitor and control access between your installation and the broader Internet. For further information including limitations of installation behind a proxy, see the Installing server behind a proxy guide.
The following fields need to be configured in your
Enter details for
proxy.https.host, along with their associated ports. These values can be the same but they both need to be configured.
For authentication you will need to configure
"1". You will also need to configure the respective username and password for both HTTP and HTTPS.
no_proxyhosts and subnets. This should include localhost, your GitHub Enterprise host (optional), the hostname of your CircleCI installation (see Known Limitations for an explanation), and the CIDRs of both vm-service and Nomad.
proxy: enabled: "1" http: host: "<proxy.example.internal>" port: "3128" auth: enabled: "1" username: "<proxy-user>" password: "<proxy-password>" https: host: "<proxy.example.internal>" port: "3128" auth: enabled: "1" username: "<proxy-user>" password: "<proxy-password>" no_proxy: - localhost - 127.0.0.1 - "<github.example.internal>" - "<circleci.example.internal>" - "<nomad-subnet-cidr>" - "<vm-service-cidr>" - "<vpc-or-subnet-cidr>" # VPC or subnets to exclude from the proxy (optional)
n. Encrypting Environment Variables
Use Tink (optional)
The following steps cover using Tink as an alternative to Vault:
Enable Tink in your
tink: enabled: false keyset: ""
tink.enabledis true, Vault will not be deployed.
Tink or Vault must be set once during installation, and cannot be changed after deployment.
tinkey create-keyset --key-template XCHACHA20_POLY1305
CircleCI server will store your generated keyset in a Kubernetes secret. You may generate this secret in either of the following ways:
|If your Tink keyset is somehow lost, you will need to generate a new keyset and then recreate your contexts and their associated secrets.|
Once you have completed the fields detailed above, you can deploy CircleCI’s core services:
USERNAME=<provided-username> PASSWORD=<token> namespace=<your-namespace> helm registry login cciserver.azurecr.io/circleci-server -u $USERNAME -p $PASSWORD helm install circleci-server oci://cciserver.azurecr.io/circleci-server -n $namespace --version 4.1.6 -f <path-to-values.yaml>
5. Create DNS entry
Create a DNS entry for your NGINX load balancer, for example,
app.circleci.your.domain.com. The DNS entry should align with the DNS names used when creating your TLS certificate and GitHub OAuth app during the prerequisites steps. All traffic will be routed through this DNS record.
You need the IP address, or, if using AWS, the DNS name of the NGINX load balancer. You can find this information with the following command:
kubectl get service circleci-proxy
You should now be able to navigate to your CircleCI server installation and log in to the application successfully.
Now we will move on to build services. It may take a while for all your services to be up. You can periodically check by running the following command (you are looking for the
frontend” pod to show a status of
running and ready should show
kubectl get pods -n <YOUR_CIRCLECI_NAMESPACE>
|VM service and Nomad server pods are expected to fail at this stage. You will set up your execution environments in the next phase of the installation.|
Help make this document better
This guide, as well as the rest of our docs, are open source and available on GitHub. We welcome your contributions.
- Suggest an edit to this page (please read the contributing guide first).
- To report a problem in the documentation, or to submit feedback and comments, please open an issue on GitHub.
- CircleCI is always seeking ways to improve your experience with our platform. If you would like to share feedback, please join our research community.
Our support engineers are available to help with service issues, billing, or account related questions, and can help troubleshoot build configurations. Contact our support engineers by opening a ticket.
You can also visit our support site to find support articles, community forums, and training resources.
CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.