This guide focuses on getting you started with the core of the CircleCI experience -
config.yml in the following simple steps:
- Part one: Hello, it’s all about the shell
- Part two: Info and preparing to build
- Part three: That’s nice but I need…
- Part four: Approved to start
Getting started with CircleCI config
This guide describes how CircleCI finds and runs
config.yml and how you can use shell commands to do things, then it outlines how
config.yml can interact with code and kick-off a build
followed by how to use docker containers to run in precisely the environment that you need. Finally, there is a short exploration of workflows so you can learn to orchestrate your build, tests, security scans, approval steps, and deployment.
CircleCI believes in configuration as code. As a result, the entire delivery process from build to deploy is orchestrated through a single file called
config.yml file is located in a folder called
.circleci at the top of your project. CircleCI uses the YAML syntax for config, see the Writing YAML document for basics.
Part one: Hello, it’s all about the shell
Let’s get started. CircleCI provides a powerful experience because we provide you an on-demand shell to run whatever you need. In this first example, we will show you how easy it is to setup your first build and execute a shell command.
- If you haven’t already, go ahead and sign-up with CircleCI and select either GitHub or Bitbucket. If you prefer, you can also sign up through the GitHub marketplace.
- Make sure the project you want to manage has been added.
- Add a
.circlecifolder at the top of your project’s master branch. If you want to experiment on a different branch, you can do that too. Make sure the folder starts with a period. This folder is special and hence the .circleci format.
- Add a
config.ymlfile inside the .circleci folder.
- Add the following contents to your
1 2 3 4 5 6 7 8 9 10 11 12 13 14 version: 2.1 jobs: build: docker: - image: alpine:3.7 auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - run: name: The First Step command: | echo 'Hello World!' echo 'This is the delivery pipeline'
Check-in the config and see it run. You can see the output of the job in the CircleCI app.
The CircleCI config syntax is very straight forward. The trickiest part is typically indentation. Make sure your indentation is consistent. This is the single most common error in config. Let’s go over the nine lines in details
- Line 1: This indicates the version of the CircleCI platform you are using.
2.1is the most recent version.
- Line 2-3: The
jobslevel contains a collection of arbitrarily named children.
buildis the first named child in the
jobscollection. In this case
buildis also the only job.
- Line 6-7: The
stepscollection is an ordered list of
rundirective is executed in the order in which it was declared.
- Line 8: The
nameattribute provides useful organizational information when returning warnings, errors, and output. The
nameshould be meaningful to you as an action within your build process
- Line 9-11: This is the magic. The
commandattribute is a list of shell commands that represent the work you want done. The initial pipe,
|, indicates that there will be more than one line of shell commands. Here line 10 will print out
Hello World!in your build shell and line 11 will print out
This is the delivery pipeline
Part two: Info and preparing to build
That was nice but let’s get real. Delivery graphs start with code. In this example we will add a few lines that will get your code and then list it. We will also do this in a second run.
If you haven’t already, please go through Part One and add a simple
.circleci/config.ymlfile to your project.
CircleCI provides a number of simplified commands that you can use to make complex interactions very easy. Here we are going to add the
checkoutcommand. This command automatically gets the branch code for you to work with in subsequent steps.
Now, add a second
runstep and do an
ls -alto see that all of your code is available.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 version: 2.1 jobs: build: docker: - image: alpine:3.7 auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - checkout - run: name: The First Step command: | echo 'Hello World!' echo 'This is the delivery pipeline' - run: name: Code Has Arrived command: | ls -al echo '^^^That should look familiar^^^'
Although we’ve only made two small changes to the config, these represent significant organizational concepts.
- Line 7: The
checkoutcommand is an example of a built-in reserved word that contextualizes your job. In this case, it is pulling down your code so you can start a build.
- Line 13-17: The second run on the
buildjob is listing (through
ls -al) the contents of the checkout. Your branch is now available for you to interact with.
Part three: That’s nice but I need…
Every code base and project is different. That’s okay. We like diversity. This is one of the reasons we allow you to run in your machine or docker container of choice. In this case we will demonstrate running in a container with node available. Other examples might include macOS machines, java containers, or even GPU.
This section expands on Part One and Two. If you haven’t already, go through at least Part One to ensure you have a working
config.ymlfile in your branch.
This is a very simple and yet amazingly powerful change. We are going to add a reference to a docker image for the build job.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 version: 2.1 jobs: build: # pre-built images: https://circleci.com/docs/2.0/circleci-images/ docker: - image: circleci/node:10-browsers auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - checkout - run: name: The First Step command: | echo 'Hello World!' echo 'This is the delivery pipeline' - run: name: Code Has Arrived command: | ls -al echo '^^^That should look familiar^^^' - run: name: Running in a Unique Container command: | node -v
We also added a small
run block that demonstrates we are running in a node container.
The above two changes to the config significantly affect how you get work done. By associating a docker container to a job and then dynamically running the job in the container, you don’t need to perform special magic or operational gymnastics to upgrade, experiment or tune the environment you run in. With a small change you can dramatically upgrade a mongo environment, grow or shrink the base image, or even change languages.
- Line 4: Here we see a comment in-line in yml. Like any other unit of code, comments are a useful tool as config gets complicated.
- Line 5-6: These lines indicate that docker image to use for the job. Because you can have more than one job in your config (as we will see next) you can also run different parts of your config in different environments. For example, you could perform a build job in a thin java container and then perform a test job using a container with browsers pre-installed. In this case, it uses a pre-built container from CircleCI that already has a browser and other useful tools built in.
- Line 19-22: These lines add a run step that returns the version of node available in the container. Try experimenting with different containers from CircleCI’s pre-built convenience images or even public containers from Docker hub.
Part four: Approved to start
So far so good? Let’s spend a moment on orchestration. In this example, we will spend more time doing analysis than step-by-step modification.
The CircleCI workflow model is based on the orchestration of predecessor jobs. This is why the reserved word used for workflow definition is
requires. Jobs initiation is always defined in terms of the successful completion of prior jobs. For example, a job vector such as [A, B, C] would be implemented with jobs B and C each requiring the job prior. Job A would not have a requires block because it starts immediately. For example, job A starts immediately; B requires A; C requires B.
In the example below, an event triggering a build will cause
Hello-World to start immediately. The remainder of the jobs will wait. When
Hello-World completes, both
Run-With-Node will start. That is because both
Hello-World to complete successfully before they can start. Next, the approval job called
Hold-For-Approval will become available when both
Run-With-Node complete. The
Hold-For-Approval job is slightly different from the others. It represents a manual intervention to allow the workflow to continue. While the workflow is waiting for a user (through the CircleCI UI or API) to approve the job, all state is preserved based on the original triggering event. CircleCI understands that Approval jobs may take hours or even days before completing - although we suggest hours over days. Once
Hold-For-Approval completes through a manual intervention, the final job
Now-Complete will run.
All of the job names are arbitrary. This allows you to create workflows as complex as you need while staying meaningful and clear to the next developer that reads the
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 version: 2.1 jobs: Hello-World: docker: - image: alpine:3.7 auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - run: name: Hello World command: | echo 'Hello World!' echo 'This is the delivery pipeline' I-Have-Code: docker: - image: alpine:3.7 auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - checkout - run: name: Code Has Arrived command: | ls -al echo '^^^That should look familiar^^^' Run-With-Node: docker: - image: circleci/node:10-browsers auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - run: name: Running In A Container With Node command: | node -v Now-Complete: docker: - image: alpine:3.7 auth: username: mydockerhub-user password: $DOCKERHUB_PASSWORD # context / project UI env-var reference steps: - run: name: Approval Complete command: | echo 'Do work once the approval has completed' workflows: version: 2 Example_Workflow: jobs: - Hello-World - I-Have-Code: requires: - Hello-World - Run-With-Node: requires: - Hello-World - Hold-For-Approval: type: approval requires: - Run-With-Node - I-Have-Code - Now-Complete: requires: - Hold-For-Approval
We now know how to create a workflow including a manual gate that you can use to protect promotion of expensive interactions.
- Line 3: The command that echo’s
Hello World!has been turned in a full fledged job with the name Hello-World
- Line 12: The commands to get code are now in a job named
- Line 22: The Node example using the CircleCI pre-built image is now called
- Line 30: There is an additional job that operates similarly to
Hello-Worldbut it won’t run until the approval is complete - see line 57 in the workflow stanza.
- Line 39-57: The config now has a workflow. In the prior examples the CircleCI engine interpreted the config as having had a single-job workflow. Here we are staying clear and spelling out the workflow we want to execute. This workflow demonstrates several useful capabilities. The
requiresstatement represents a list of prior jobs that must complete successfully prior to the job in question starting. In this example, both
Run-With-Nodemust complete before
Hold-For-Approvalbecomes active. In addition, both
Run-With-Nodeare dependent on
Hello-Worldbut not on each other. This means that those two jobs will run concurrently as soon as
Hello-Worldis complete. This is useful if you have multiple jobs that are not directly dependent on one another and you want to improve wall-clock time.
- Line 50-51: Most of the jobs are generic. However, this job has a type. In this case the type is
approvaland requires a person through the CircleCI API or UI to take an action for the build to complete. Interleaving approval jobs allows you to create gates that must be approved or managed prior to downstream jobs executing.
The examples above were designed to provide a quick starter to the areas of functionality available through CircleCI config. There remains a lot more. Take a look at the rest of the documentation. You will find that scheduled jobs, workspaces, artifacts, and more are all simple variations on the concepts you’ve learned here. Now go forth and automate your CI/CD world!