Start Building for Free
CircleCI.comAcademyBlogCommunitySupport

Operator overview

8 months ago1 min read
Server v4.1
Server Admin
On This Page
  • Execution environment
  • Nomad clients
  • GitHub
  • Next steps

This guide contains information for CircleCI server operators, or those responsible for ensuring CircleCI server is running properly through maintenance and monitoring. Before reading this operator guide, ensure you have read the CircleCI server v4.1 overview.

CircleCI server schedules CI/CD jobs using the Nomad scheduler. The Nomad Server (control plane) runs inside the Kubernetes cluster, while Nomad clients are provisioned outside the cluster. The Nomad clients need access to the Nomad control plane, output processor, and VM service.

CircleCI server can run Docker jobs on the Nomad clients, and also in dedicated virtual machines. These VM jobs are controlled by the Nomad clients, therefore the Nomad clients must be able to access the VMs on port 22 for SSH and port 2376 for remote Docker jobs.

Job artifacts and outputs are sent directly from jobs in Nomad to object storage (S3, GCS, or other supported options).

Audit logs and other items from the application are also stored in object storage, so both the Kubernetes cluster and the Nomad clients need access to object storage.

Execution environment

CircleCI server v4.x uses Nomad as the primary job scheduler. Refer to our Introduction to Nomad Cluster Operation to learn more about the job scheduler and how to perform basic client and cluster operations.

CircleCI Nomad clients automatically provision compute resources according to the executors configured for each job in a project’s .circleci/config.yml file.

Nomad clients

Nomad Clients run without storing state, allowing you to increase or decrease the number of containers as needed.

If a job’s resource class requires more resources than the Nomad client’s instance type has available, it will remain in a pending state. Choosing a smaller instance type for Nomad clients is a way to reduce cost, but limits the Docker resource classes CircleCI can use. Review the available resource classes to decide what is best for you. The default instance type will run up to xlarge resource classes.

See the Nomad Documentation for options for optimizing the resource usage of Nomad clients.

For more information on Nomad port requirements, see the Hardening Your Cluster guide.

GitHub

CircleCI uses GitHub or GitHub Enterprise as an identity provider. GitHub Enterprise can, in turn, use SAML or SCIM to manage users from an external identity provider.

The following table describes the ports used on machines running GitHub to communicate with the services and Nomad client instances.

SourcePortsUse

Services

22

Git access

Services

80 or 443

API access

Nomad Client

22

Git access

Nomad Client

80 or 443

API access


Suggest an edit to this page

Make a contribution
Learn how to contribute