CircleCI Server v3.x VM Service

Last updated
Tags Server v3.x Server Admin

CircleCI server’s VM service controls how machine executor (Linux and Windows images) and Remote Docker jobs are run.

This section describes the available configuration options for VM service. These config options are all accessible from the KOTS admin console by choosing the Config tab from your dashboard.

We recommend that you leave these options at their defaults until you have successfully configured and verified the core and build services of your server installation. Steps to set up VM service are provided in the server 3.x installation guide for AWS and GCP.

VM service settings

You will need to provide the hostname for your VM service load balancer:

  1. Once your server installation is up and running, run the following:

    kubectl get svc/vm-service
  2. Enter the address listed under External IP in the VM Service Load Balancer Hostname field.

There is also an option to change the port used for VM service. The default is 3000 and this should only be changed if you are guided to do so by a CircleCI support engineer.

VM provider

The following configuration options are for the VM provider: either AWS or GCP.

AWS EC2

You will need to complete the following fields to configure your VM Service to work with AWS EC2.

At this point you can uncheck the Assign Public IPs check box if you need VMs to use private IP addresses.

  • AWS Region (required): This is the region the application is in.

  • AWS Linux AMI ID (optional): If you wish to provide a custom AMI for Linux machine executors you can supply an AMI ID here. To create a Linux image use the CircleCI Server Linux Image Builder. If you leave this field blank, a default AMI will be used.

  • AWS Windows AMI ID (optional): If you require Windows executors, you can supply an AMI ID for them here. To create a Windows image, use the CircleCI Server Windows Image Builder. Leave this field blank if you do not require them.

  • Subnets (required): Choose subnets (public or private) where the VMs should be deployed. Note that all subnets must be in the same availability zone.

  • Security Group ID (required): This is the security group that will be attached to the VMs.

    The recommended security group configuration can be found in the Hardening Your Cluster section.

  • AWS IAM Access Key ID (required): AWS Access Key ID for EC2 access.

    The Access Key and Secret Key used by VM Service differs from the policy used by object storage in the previous section.
  • AWS IAM Secret Key (required): IAM Secret Key for EC2 access.

  • Number of <VM-type> VMs to keep prescaled: By default, this field is set to 0 which will create and provision instances of a resource type on demand. You have the option of preallocating up to 5 instances per resource type. Preallocating instances lowers the start time allowing for faster machine and remote_docker builds.

    preallocated instances are always running and could potentially increase costs. Decreasing this number may also take up to 24 hours for changes to take effect. You have the option of terminating those instances manually, if required.
    If Docker Layer Caching (DLC) is to be used, VM Service instances need to be spun up on-demand. To ensure this can happen, either ensure any preallocated instances are in use, or set both remote Docker and machine preallocated instance fields to 0.
    When using preallocated instances be aware that a cron job is scheduled to cycle through these instances once per day to ensure they don’t end up in an unworkable state.

Default AWS AMI list

The default AMIs for server v3.x are based on Ubuntu 20.04.

"us-east-1" "ami-04f249339fa8afc90"
"ca-central-1" "ami-002f61fb4f6cd4f04"
"ap-south-1" "ami-0309e6438340ff3f5"
"ap-southeast-2" "ami-03ac956e1d298b76a"
"ap-southeast-1" "ami-0272b002478c96552"
"eu-central-1" "ami-07266a91e4ef7e3e8"
"eu-west-1" "ami-0bc8a965f9ae82e44"
"eu-west-2" "ami-0bcbed1cffe3866c2"
"sa-east-1" "ami-05291e231356c0387"
"us-east-2" "ami-08735066168c5c8e9"
"us-west-1" "ami-035e0e862838fcb21"
"us-west-2" "ami-0b4970c467d8baaef"
"ap-northeast-1" "ami-0b9233227f60abc2c"
"ap-northeast-2" "ami-08e7a9df6ab2f6b9d"
"eu-west-3" "ami-07f0d51c7621f0c39"
"us-gov-east-1" "ami-0f68718afd37587ae"
"us-gov-west-1" "ami-8e2106ef"

Google Cloud Platform

You will need the following fields to configure your VM service to work with Google Cloud Platform (GCP).

We recommend you create a unique service account used exclusively by VM Service. The Compute Instance Admin (Beta) role is broad enough to allow VM Service to operate. If you wish to make permissions more granular, you can use the Compute Instance Admin (beta) role documentation as reference.

At this point you can uncheck the Assign Public IPs check box if you need VMs to use private IP addresses.

  • GCP project ID (required): Name of the GCP project the cluster resides.

  • GCP Zone for your VMs (required): GCP zone the virtual machines instances should be created in IE us-east1-b.

  • GCP Windows Image (optional): If you require Windows executors, you can supply an AMI ID for them here. To create a Windows image, use the CircleCI Server Windows Image Builder. Leave this field blank if you do not require them.

  • GCP VPC Network (required): Name of the VPC Network.

  • GCP VPC Subnet (optional): Name of the VPC Subnet. If using auto-subnetting, leave this field blank.

  • GCP Service Account JSON file (required): Copy and paste the contents of your service account JSON file.

  • Number of <VM-type> VMs to keep prescaled: By default, this field is set to 0 which will create and provision instances of a resource type on demand. You have the option of preallocating up to 5 instances per resource type. Preallocating instances lowers the start time allowing for faster machine and remote_docker builds.

    preallocated instances are always running and could potentially increase costs. Decreasing this number may also take up to 24 hours for changes to take effect. You have the option of terminating those instances manually, if required.
    If Docker Layer Caching (DLC) is to be used, VM Service instances need to be spun up on-demand. To ensure this can happen, either ensure any preallocated instances are in use, or set both remote Docker and machine preallocated instance fields to 0.
    When using preallocated instances be aware that a cron job is scheduled to cycle through these instances once per day to ensure they don’t end up in an unworkable state.


Help make this document better

This guide, as well as the rest of our docs, are open-source and available on GitHub. We welcome your contributions.