1. cci-labs/on-demand-runner@1.0.0

cci-labs/on-demand-runner@1.0.0

Sections
A orb that lets users spin up runners on demand durning a pipeline. Users are able to provision an EC2 instance on the fly, install the runner agent, and allow the runner to accept units of work.
Created: April 1, 2022Version Published: April 11, 2022Releases: 1
Org Usage:
< 25

Orb Quick Start Guide

Use CircleCI version 2.1 at the top of your .circleci/config.yml file.

1 version: 2.1

Add the orbs stanza below your version, invoking the orb:

1 2 orbs: on-demand-runner: cci-labs/on-demand-runner@1.0.0

Use on-demand-runner elements in your existing workflows and jobs.

Opt-in to use of uncertified orbs on your organization’s Security settings page.

Usage Examples

run-test-on-runner

Deploy an On Demand Runner, Do Testing on the Runner and Destroy the On Demand Runner

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 version: '2.1' orbs: on-demand-runner: cci-labs/on-demand-runner@1.0.0 jobs: run-test-on-runner: machine: true resource_class: NAMESPACE/<< pipeline.id >> steps: - run: echo "Hello World! I am running on an On Demand Runner!" workflows: example-workflow: jobs: - on-demand-runner/deploy: context: aws-creds resource_class: << pipeline.id >> - run-test-on-runner: requires: - on-demand-runner/deploy - on-demand-runner/destroy: context: aws-creds requires: - run-test-on-runner resource_class: << pipeline.id >>

Jobs

deploy

Provision and Configure an On Demand Runner Required Environment Variables: - AWS_AMI_ID - AWS_SECURITY_GROUP_ID - AWS_SSH_KEY - CIRCLECI_NAMESPACE - CIRCLE_TOKEN - AWS_ACCESS_KEY_ID - AWS_SECRET_ACCESS_KEY

Show job Source
PARAMETER
DESCRIPTION
REQUIRED
DEFAULT
TYPE
aws-access-key-id
AWS access key id for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_ACCESS_KEY_ID.
No
AWS_ACCESS_KEY_ID
env_var_name
aws-ami-id
An AWS AMI(Amazon Machine Image) ID for the image you want spun up as an EC2 instance. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_AMI_ID.
No
AWS_AMI_ID
env_var_name
aws-instance-type
A valid instance type/size for a EC2 instance. If no instance type is provided a t2.mirco will be selected. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_INSTANCE_TYPE.
No
AWS_INSTANCE_TYPE
env_var_name
aws-region
Env var of AWS region to operate in. If no region is provided US-EAST-1 will be selected.
No
AWS_DEFAULT_REGION
env_var_name
aws-secret-access-key
AWS secret key for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. $AWS_SECRET_ACCESS_KEY.
No
AWS_SECRET_ACCESS_KEY
env_var_name
aws-security-group-id
The ID of the AWS security group you want to be applied to the EC2 instance(s). Set this to the name of the environment variable you will use to hold this value, i.e. AWS_SECURITY_GROUP_ID.
No
AWS_SECURITY_GROUP_ID
env_var_name
aws-ssh-key
The name of the SSH key you want the EC2 instance(s) to have injected. Please note you need to already have uploaded the key to AWS to reference the SSH key name. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_SSH_KEY.
No
AWS_SSH_KEY
env_var_name
aws-vpc-id
The ID of the VPC you want the EC2 instance(s) to be deployed in. If left blank, the default VPC will be selected. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_VPC_ID.
No
AWS_VPC_ID
env_var_name
circle-token
A valid Circle token for interacting with the CircleCI API. Please see https://circleci.com/docs/2.0/managing-api-tokens/ for more details.
No
CIRCLE_TOKEN
env_var_name
circleci-namespace
A CircleCI generated namespace for your organization. Please see https://circleci.com/docs/2.0/orb-concepts/#namespaces for more details.
No
CIRCLECI_NAMESPACE
env_var_name
number_of_runners
Number of on demand runners to deploy. If no number is provided only 1 on demand runner will be deployed. PLEASE be careful setting this number.
No
NUMBER_OF_RUNNERS
env_var_name
number_of_waits
Number of on waits(10 seconds each) for the EC2 instance(s) to be provisioned before failure. If no number is provided 18 waits(3 minutes) will be selected.
No
NUMBER_OF_WAITS
env_var_name
resource_class
Name of the resource class that will be generated and used for On Demand Runners. If you do not have a resource class to use, please pass in 'pipeline.id'. Generally, it best practice to pass in 'pipeline.id'. PLEASE note you need to set resource class to a name that is not currently being used, as the resource class will be deleted.
Yes
-
string
runner_user
The username the runner agent will utilize to run its service. If no username is given 'circleci' will be used.
No
RUNNER_USER
env_var_name
tag
The `cimg/python` Docker image version tag.
No
'3.10'
string
version
Ansible Version To Be Installed
No
2.9.*
string

destroy

Deprovision and Remove an On Demand Runner Required Environment Variables: - AWS_SSH_KEY - CIRCLECI_NAMESPACE - CIRCLE_TOKEN - AWS_ACCESS_KEY_ID - AWS_SECRET_ACCESS_KEY

Show job Source
PARAMETER
DESCRIPTION
REQUIRED
DEFAULT
TYPE
aws-access-key-id
AWS access key id for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_ACCESS_KEY_ID.
No
AWS_ACCESS_KEY_ID
env_var_name
aws-region
Env var of AWS region to operate in. If no region is provided US-EAST-1 will be selected.
No
AWS_DEFAULT_REGION
env_var_name
aws-secret-access-key
AWS secret key for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. $AWS_SECRET_ACCESS_KEY.
No
AWS_SECRET_ACCESS_KEY
env_var_name
aws-ssh-key
The name of the SSH key you want the EC2 instance(s) to have injected. Please note you need to already have uploaded the key to AWS to reference the SSH key name. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_SSH_KEY.
No
AWS_SSH_KEY
env_var_name
circle-token
A valid Circle token for interacting with the CircleCI API. Please see https://circleci.com/docs/2.0/managing-api-tokens/ for more details.
No
CIRCLE_TOKEN
env_var_name
circleci-namespace
A CircleCI generated namespace for your organization. Please see https://circleci.com/docs/2.0/orb-concepts/#namespaces for more details.
No
CIRCLECI_NAMESPACE
env_var_name
number_of_runners
Number of on demand runners to deploy. If no number is provided only 1 on demand runner will be deployed. PLEASE be careful setting this number.
No
NUMBER_OF_RUNNERS
env_var_name
number_of_waits
Number of on waits(10 seconds each) for the EC2 instance(s) to be provisioned before failure. If no number is provided 18 waits(3 minutes) will be selected.
No
NUMBER_OF_WAITS
env_var_name
resource_class
Name of the resource class that will be generated and used for an On Demand Runner. If you do not have a resource class to use, please pass in 'pipeline.id'. Generally, it best practice to pass in 'pipeline.id'. PLEASE note you need to set resource class to a name that is not currently being used, as the resource class will be deleted.
Yes
-
string
tag
The `cimg/python` Docker image version tag.
No
'3.10'
string
version
Ansible Version To Be Installed
No
2.9.*
string

Orb Source

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 # This code is licensed from CircleCI to the user under the MIT license. # See here for details: https://circleci.com/developer/orbs/licensing version: 2.1 description: | A orb that lets users spin up runners on demand durning a pipeline. Users are able to provision an EC2 instance on the fly, install the runner agent, and allow the runner to accept units of work. display: home_url: https://github.com/CircleCI-Public/on-demand-runner-orb source_url: https://github.com/CircleCI-Public/on-demand-runner-orb jobs: deploy: description: | Provision and Configure an On Demand Runner Required Environment Variables: - AWS_AMI_ID - AWS_SECURITY_GROUP_ID - AWS_SSH_KEY - CIRCLECI_NAMESPACE - CIRCLE_TOKEN - AWS_ACCESS_KEY_ID - AWS_SECRET_ACCESS_KEY docker: - image: cimg/python:<< parameters.tag >> parameters: aws-access-key-id: default: AWS_ACCESS_KEY_ID description: | AWS access key id for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_ACCESS_KEY_ID. type: env_var_name aws-ami-id: default: AWS_AMI_ID description: | An AWS AMI(Amazon Machine Image) ID for the image you want spun up as an EC2 instance. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_AMI_ID. type: env_var_name aws-instance-type: default: AWS_INSTANCE_TYPE description: | A valid instance type/size for a EC2 instance. If no instance type is provided a t2.mirco will be selected. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_INSTANCE_TYPE. type: env_var_name aws-region: default: AWS_DEFAULT_REGION description: | Env var of AWS region to operate in. If no region is provided US-EAST-1 will be selected. type: env_var_name aws-secret-access-key: default: AWS_SECRET_ACCESS_KEY description: | AWS secret key for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. $AWS_SECRET_ACCESS_KEY. type: env_var_name aws-security-group-id: default: AWS_SECURITY_GROUP_ID description: | The ID of the AWS security group you want to be applied to the EC2 instance(s). Set this to the name of the environment variable you will use to hold this value, i.e. AWS_SECURITY_GROUP_ID. type: env_var_name aws-ssh-key: default: AWS_SSH_KEY description: | The name of the SSH key you want the EC2 instance(s) to have injected. Please note you need to already have uploaded the key to AWS to reference the SSH key name. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_SSH_KEY. type: env_var_name aws-vpc-id: default: AWS_VPC_ID description: | The ID of the VPC you want the EC2 instance(s) to be deployed in. If left blank, the default VPC will be selected. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_VPC_ID. type: env_var_name circle-token: default: CIRCLE_TOKEN description: | A valid Circle token for interacting with the CircleCI API. Please see https://circleci.com/docs/2.0/managing-api-tokens/ for more details. type: env_var_name circleci-namespace: default: CIRCLECI_NAMESPACE description: | A CircleCI generated namespace for your organization. Please see https://circleci.com/docs/2.0/orb-concepts/#namespaces for more details. type: env_var_name number_of_runners: default: NUMBER_OF_RUNNERS description: | Number of on demand runners to deploy. If no number is provided only 1 on demand runner will be deployed. PLEASE be careful setting this number. type: env_var_name number_of_waits: default: NUMBER_OF_WAITS description: | Number of on waits(10 seconds each) for the EC2 instance(s) to be provisioned before failure. If no number is provided 18 waits(3 minutes) will be selected. type: env_var_name resource_class: description: | Name of the resource class that will be generated and used for On Demand Runners. If you do not have a resource class to use, please pass in 'pipeline.id'. Generally, it best practice to pass in 'pipeline.id'. PLEASE note you need to set resource class to a name that is not currently being used, as the resource class will be deleted. type: string runner_user: default: RUNNER_USER description: | The username the runner agent will utilize to run its service. If no username is given 'circleci' will be used. type: env_var_name tag: default: "3.10" description: The `cimg/python` Docker image version tag. type: string version: default: 2.9.* description: Ansible Version To Be Installed type: string steps: - run: command: |- #!/bin/bash pip install "ansible==${ORB_PARAM_ANSIBLE_VERSION}" pip install boto3 botocore --upgrade ansible-galaxy collection install amazon.aws community.general environment: ORB_PARAM_ANSIBLE_VERSION: <<parameters.version>> name: Installing Dependencies - run: command: | { echo "export AWS_DEFAULT_REGION=$(echo ${<< parameters.aws-region >>:-us-east-1})" echo "export AWS_INSTANCE_TYPE=$(echo ${<< parameters.aws-instance-type >>:-t2.micro})" echo "export RESOURCE_CLASS=$(echo ${RESOURCE_CLASS:-<< parameters.resource_class >>})" echo "export NUMBER_OF_RUNNERS=$(echo ${<< parameters.number_of_runners >>:-1})" echo "export NUMBER_OF_WAITS=$(echo ${<< parameters.number_of_waits >>:-18})" echo "export ANSIBLE_FORCE_COLOR=$(echo ${ANSIBLE_FORCE_COLOR:-true})" echo "export ANSIBLE_GIT_URL=$(echo ${ANSIBLE_GIT_URL:-https://github.com/CircleCI-Public/on-demand-runner-orb.git})" echo "export ANSIBLE_GIT_BRANCH=$(echo ${ANSIBLE_GIT_BRANCH:-main})" echo "export ANSIBLE_HOST_KEY_CHECKING=$(echo ${ANSIBLE_HOST_KEY_CHECKING:-false})" echo "export RESOURCE_CLASS_CREATION=$(echo ${RESOURCE_CLASS_CREATION:-true})" echo "export RUNNER_TOKEN_CREATION=$(echo ${RUNNER_TOKEN_CREATION:-true})" echo "export TEMPLATE_DYNAMIC_INVENTORY=$(echo ${TEMPLATE_DYNAMIC_INVENTORY:-true})" echo "export CLOUD_PROVIDER=$(echo ${CLOUD_PROVIDER:-aws})" echo "export CIRCLECI_USER=$(echo ${<< parameters.runner_user >>:-circleci})" } >> $BASH_ENV name: Setting Defaults if Variables are not Set - run: command: | git clone ${ANSIBLE_GIT_URL} && cd ./on-demand-runner-orb && git checkout ${ANSIBLE_GIT_BRANCH} name: Clone On Demand Runner Ansible from Github - run: command: | ansible-playbook provision-on-demand-runner.yml -e "REGION=$AWS_DEFAULT_REGION AMI_ID=$<< parameters.aws-ami-id >> SECURITY_GROUP=$<< parameters.aws-security-group-id >> VPC_ID=$<< parameters.aws-vpc-id >> SSH_KEY_NAME=$<< parameters.aws-ssh-key >> INSTANCE_TYPE=$AWS_INSTANCE_TYPE NAMESPACE=$<< parameters.circleci-namespace >> RESOURCE_CLASS=$RESOURCE_CLASS NUMBER_OF_RUNNERS=$NUMBER_OF_RUNNERS UNTIL_RETRIES_RUNNERS=$NUMBER_OF_WAITS TEMPLATE_DYNAMIC_INVENTORY=$TEMPLATE_DYNAMIC_INVENTORY CLOUD_PROVIDER=$CLOUD_PROVIDER" name: Provision an On Demand Runner working_directory: ~/project/on-demand-runner-orb/auto-runner/ - run: command: | ansible-playbook -i aws_ec2.yml install-runner.yml -u ansible -e "NAMESPACE=$<< parameters.circleci-namespace >> RESOURCE_CLASS=$RESOURCE_CLASS CIRCLE_TOKEN=$<< parameters.circle-token >> RESOURCE_CLASS_CREATION=$RESOURCE_CLASS_CREATION RUNNER_TOKEN_CREATION=$RUNNER_TOKEN_CREATION target_hosts=all CIRCLECI_USER=$CIRCLECI_USER" name: Configure the On Demand Runner working_directory: ~/project/on-demand-runner-orb/auto-runner/ destroy: description: | Deprovision and Remove an On Demand Runner Required Environment Variables: - AWS_SSH_KEY - CIRCLECI_NAMESPACE - CIRCLE_TOKEN - AWS_ACCESS_KEY_ID - AWS_SECRET_ACCESS_KEY docker: - image: cimg/python:<< parameters.tag >> parameters: aws-access-key-id: default: AWS_ACCESS_KEY_ID description: | AWS access key id for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_ACCESS_KEY_ID. type: env_var_name aws-region: default: AWS_DEFAULT_REGION description: | Env var of AWS region to operate in. If no region is provided US-EAST-1 will be selected. type: env_var_name aws-secret-access-key: default: AWS_SECRET_ACCESS_KEY description: | AWS secret key for IAM role. Set this to the name of the environment variable you will use to hold this value, i.e. $AWS_SECRET_ACCESS_KEY. type: env_var_name aws-ssh-key: default: AWS_SSH_KEY description: | The name of the SSH key you want the EC2 instance(s) to have injected. Please note you need to already have uploaded the key to AWS to reference the SSH key name. Set this to the name of the environment variable you will use to hold this value, i.e. AWS_SSH_KEY. type: env_var_name circle-token: default: CIRCLE_TOKEN description: | A valid Circle token for interacting with the CircleCI API. Please see https://circleci.com/docs/2.0/managing-api-tokens/ for more details. type: env_var_name circleci-namespace: default: CIRCLECI_NAMESPACE description: | A CircleCI generated namespace for your organization. Please see https://circleci.com/docs/2.0/orb-concepts/#namespaces for more details. type: env_var_name number_of_runners: default: NUMBER_OF_RUNNERS description: | Number of on demand runners to deploy. If no number is provided only 1 on demand runner will be deployed. PLEASE be careful setting this number. type: env_var_name number_of_waits: default: NUMBER_OF_WAITS description: | Number of on waits(10 seconds each) for the EC2 instance(s) to be provisioned before failure. If no number is provided 18 waits(3 minutes) will be selected. type: env_var_name resource_class: description: | Name of the resource class that will be generated and used for an On Demand Runner. If you do not have a resource class to use, please pass in 'pipeline.id'. Generally, it best practice to pass in 'pipeline.id'. PLEASE note you need to set resource class to a name that is not currently being used, as the resource class will be deleted. type: string tag: default: "3.10" description: The `cimg/python` Docker image version tag. type: string version: default: 2.9.* description: Ansible Version To Be Installed type: string steps: - run: command: |- #!/bin/bash pip install "ansible==${ORB_PARAM_ANSIBLE_VERSION}" pip install boto3 botocore --upgrade ansible-galaxy collection install amazon.aws community.general environment: ORB_PARAM_ANSIBLE_VERSION: <<parameters.version>> name: Installing Dependencies - run: command: | { echo "export AWS_DEFAULT_REGION=$(echo ${<< parameters.aws-region >>:-us-east-1})" echo "export RESOURCE_CLASS=$(echo ${RESOURCE_CLASS:-<< parameters.resource_class >>})" echo "export NUMBER_OF_RUNNERS=$(echo ${<< parameters.number_of_runners >>:-1})" echo "export NUMBER_OF_WAITS=$(echo ${<< parameters.number_of_waits >>:-18})" echo "export ANSIBLE_FORCE_COLOR=$(echo ${ANSIBLE_FORCE_COLOR:-true})" echo "export ANSIBLE_GIT_URL=$(echo ${ANSIBLE_GIT_URL:-https://github.com/CircleCI-Public/on-demand-runner-orb.git})" echo "export ANSIBLE_GIT_BRANCH=$(echo ${ANSIBLE_GIT_BRANCH:-main})" echo "export ANSIBLE_HOST_KEY_CHECKING=$(echo ${ANSIBLE_HOST_KEY_CHECKING:-false})" echo "export RESOURCE_CLASS_CREATION=$(echo ${RESOURCE_CLASS_CREATION:-true})" echo "export RUNNER_TOKEN_CREATION=$(echo ${RUNNER_TOKEN_CREATION:-true})" echo "export TEMPLATE_DYNAMIC_INVENTORY=$(echo ${TEMPLATE_DYNAMIC_INVENTORY:-true})" echo "export CLOUD_PROVIDER=$(echo ${CLOUD_PROVIDER:-aws})" } >> $BASH_ENV name: Setting Defaults if Variables are not Set - run: command: | git clone ${ANSIBLE_GIT_URL} && cd ./on-demand-runner-orb && git checkout ${ANSIBLE_GIT_BRANCH} name: Clone On Demand Runner Ansible from Github - run: command: | ansible-playbook deprovision-on-demand-runner.yml -e "REGION=$AWS_DEFAULT_REGION SSH_KEY_NAME=$<< parameters.aws-ssh-key >> NAMESPACE=$<< parameters.circleci-namespace >> RESOURCE_CLASS=$RESOURCE_CLASS CIRCLE_TOKEN=$<< parameters.circle-token >> NUMBER_OF_RUNNERS=$NUMBER_OF_RUNNERS UNTIL_RETRIES_RUNNERS=$NUMBER_OF_WAITS RESOURCE_CLASS_CREATION=$RESOURCE_CLASS_CREATION RUNNER_TOKEN_CREATION=$RUNNER_TOKEN_CREATION TEMPLATE_DYNAMIC_INVENTORY=$TEMPLATE_DYNAMIC_INVENTORY CLOUD_PROVIDER=$CLOUD_PROVIDER" name: Deprovision the On Demand Runner working_directory: ~/project/on-demand-runner-orb/auto-runner/ examples: run-test-on-runner: description: | Deploy an On Demand Runner, Do Testing on the Runner and Destroy the On Demand Runner usage: version: "2.1" orbs: on-demand-runner: cci-labs/on-demand-runner@1.0.0 jobs: run-test-on-runner: machine: true resource_class: NAMESPACE/<< pipeline.id >> steps: - run: echo "Hello World! I am running on an On Demand Runner!" workflows: example-workflow: jobs: - on-demand-runner/deploy: context: aws-creds resource_class: << pipeline.id >> - run-test-on-runner: requires: - on-demand-runner/deploy - on-demand-runner/destroy: context: aws-creds requires: - run-test-on-runner resource_class: << pipeline.id >>
Developer Updates
Get tips to optimize your builds
Or join our research panel and give feedback
By submitting this form, you are agreeing to ourTerms of UseandPrivacy Policy.