Skip to content
Docs
Cancel

VM Service

This section outlines how to set up and customize VM service for your CircleCI installation. VM Service controls how machine executor (Linux and Windows images) and remote Docker jobs are run.

The information on this page is only applicable to installations on AWS. Please contact your CircleCI account representative to request VM service guidance for a static installation.

Overview

VM service enables users of CircleCI Server, installed on AWS, to run jobs using the Remote Docker Environment and the machine executor.

Configuring VM Service on CircleCI Server
Figure 1. VM Service Settings
Any changes to management console settings require downtime while the CircleCI application restarts.

The following sections will run through the settings and options displayed in the VM Service screenshot show above.

To configure VM service, it is best practice to select the AWS EC2 option in the Management Console Settings. This will allow CircleCI to run remote Docker and machine executor jobs using dedicated EC2 instances.

1. Supply AMIs

You can provide custom Amazon Machine Image (AMIs) for VM service, as described in the sections below. If you do not provide any custom images, all machine executor and remote Docker jobs will be run on instances built with one of our default AMIs (listed below), which have Ubuntu 16.04, Docker version 18.06.3 and a selection of common languages, tools, and frameworks. See the picard-vm-image branch of our image-builder repository for details. To run Windows jobs you must supply a Windows AMI, without this Windows jobs will fail to run.

Default VM service Linux AMIs

  • Ap-northeast-1:ami-0e49af0659db9fc5d

  • Ap-northeast-2:ami-03e485694bc2da249

  • Ap-south-1:ami-050370e57dfc6574a

  • Ap-southeast-1:ami-0a75ff7b28897268c

  • Ap-southeast-2:ami-072b1b45245549586

  • Ca-central-1:ami-0e44086f0f518ad2d

  • Eu-central-1:ami-09cbcfe446101b4ea

  • Eu-west-1:ami-0d1cbc2cc3075510a

  • Eu-west-2:ami-0bd22dcdc30fa260b

  • Sa-east-1:ami-038596d5a4fc9893b

  • Us-east-1:ami-0843ca047684abe87

  • Us-east-2:ami-03d60a35576647f63

  • Us-west-1:ami-06f6efb13d9ccf93d

  • Us-west-2:ami-0b5b8ad02f405a909

Customizing and Creating VM Service Images

Customizing the VM service images for your installation will allow you to specify versions of Docker and Docker Compose, as well as install any additional dependencies that may be part of your CI/CD pipeline. You can create separate AMIs for jobs that use remote Docker or the machine executor, and for machine you can specify separate AMIs for Linux and Windows. It’s worth noting that if you choose not to customize the base Linux image, developers will likely need to configure jobs to run additional install and update steps on every commit as part of each project’s config.yml file.

From Server v2.18, you can either provide a single custom Linux AMI to use for both machine and remote Docker jobs using just the field marked '1' below, or, by providing a second custom AMI in the field marked '2', you can use different settings for each.

Custom VM Service Images
Figure 2. Custom VM Service Images

Custom Linux AMI

Prerequisites

Creating a Custom Linux AMI

  1. Clone our image builder repo: https://github.com/circleci/image-builder/tree/picard-vm-image

  2. Open aws-vm.json in your editor. This provides a baseline template for building an AMI with Packer. An AWS access key ID and secret access key are required to upload. You can find more information about managing AWS authentication with Packer here. If the baseline template is too limited, you can find additional AWS configuration options here.

  3. (Optional) Consider restricting the ami_groups to only within your organization. See the [Packer documentation](https://packer.io/docs/builders/amazon-ebs.html#ami_groups) for more information on AMI groups.

  4. We provide a list of [pre-configured dependencies](https://github.com/circleci/image-builder/blob/picard-vm-image/provision.sh). You can customize the provision.sh script to meet the needs of your environment.

  5. Run packer build aws-vm.json

Once your AMI(s) have been created, copy the AMI ID(s) into the relevant field shown in the screenshot above.

Creating a Windows AMI

Introduced in CircleCI Server v2.18.3

Creating a Windows image and specifying it under the VM Service settings lets your users run jobs on dedicated Windows VMs. To create your Windows image run through the steps listed in our image builder repo, then copy the generated AMI ID and paste into the Custom Windows VM AMI field in your Management Console settings, under VM Provider (for example, <your-hostname.com:8800/settings>).

Windows images are built on CircleCI, so we suggest you run through this process once your installation is up and running. Alternatively you can use any other CircleCI account – including on our managed Cloud service – to generate the image.

2. Define Instance Types

There are two fields for defining the AWS instance types you wish to use. The first is for the default instance type, and the second is to set the instance type to use when a Job specifies the large resource class.

3. On Demand and Preallocated Instances

Remote Docker and machine executor instances are spun up on demand. It is also possible to preallocate instances to remain up and running, ready for remote Docker and machine jobs to be run (see the last two fields in figure 1).

If Docker Layer Caching (DLC) is to be used, VM Service instances need to be spun up on-demand. To ensure this can happen, either ensure any preallocated instances are in use, or set both remote Docker and machine preallocated instance fields to 0.
When using preallocated instances be aware that a cron job is scheduled to cycle through these instances once per day to ensure they don’t end up in an unworkable state.

Job and Instance Management

Jobs run using the remote Docker Environment, or the machine executor are scheduled and dispatched by the Nomad server to your Nomad clients and passed on to remote Docker or machine from there. This means jobs run on remote Docker and the machine executor can be monitored in the usual way, using the Nomad CLI. See our Introduction to Nomad Cluster Operation for more about Nomad commands and terminology.

A cron job is scheduled to cycle all default and preallocated instances at least once per day to ensure instances do not end up in an unworkable state.

Accessing Remote Docker and machine instances

By default, private IP addresses are used to communicate with VM service instances. If you need to grant wider access, for example, to allow developers SSH access, this can be set using the checkbox in the VM Provider Advanced Settings.

VM Provider Advanced Settings
Figure 3. Allowing Access to VM Service Instances

Help make this document better

This guide, as well as the rest of our docs, are open-source and available on GitHub. We welcome your contributions.

Creative Commons License
CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.