Docker and CI/CD tutorial: a deep dive into containers
Developer Advocate, CircleCI
This is a follow-up to a previous post that I wrote after CircleCI 2.0 was released. This release implemented build support using Docker executors. After the release, we realized that one of the biggest barriers that CircleCI users encountered was a lack of experience with Docker. This was confirmed through my conversations with numerous developers, in the community and at events, about their use of container technologies in their continuous integration pipelines. These conversations have highlighted that many developers don’t have a full understanding of how to use container technologies.
In this post, I will expand on and demonstrate some of the more useful Docker commands discussed in my previous post.
What is Docker?
Here is the simplified definition of Docker that I used in my previous post:
Docker is a platform for developers and sysadmins to develop, deploy, and run applications using containers.
Docker is also referred to as an application packaging tool that enables applications to be configured and packaged into a Docker image that can be used to spawn Docker containers that run instances of the application. It provides many benefits including runtime environment isolation, consistency via code, and portability. Docker containers can run on any operating system that supports the Docker Engine.
Basic Docker terminology
Here’s a list of basic Docker commands and terms with links to more information. These will help you understand Docker and control the executor. The commands can be run locally on any computer that has the Docker engine installed.
Building Docker images
Dockerfilea text document that contains all the commands a user could call on the command line to assemble an image.
The Dockerfile is a blueprint for building Docker images. Dockerfile templates hold elements such as the base operating system image used as a foundation, execution commands that install/configure dependencies, and copy commands that push local source code or artifacts into the target Docker image. Below is an example of a Dockerfile for a simple Node.js application:
FROM node:10
# Create app directory
WORKDIR /usr/src/app
# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm@5+)
COPY package*.json ./
RUN npm install --only=production
# If you are building your code for production
# RUN npm install --only=production
# Bundle app source
COPY . .
EXPOSE 5000
CMD [ "npm", "start" ]
docker imageslists Docker images found locally.
This command is used to list all of the Docker images and related data that currently exist on the local machine. It’s comparable to a linux ls or windows dir command used to show the contents of directories in a terminal. This command is very useful for understanding how to manage, maintain, and build Docker images locally. Below is an example result of a docker images execution:
REPOSITORY TAG IMAGE ID CREATED SIZE
ariv3ra/nodejs-circleci latest f419e4a6b1b8 11 days ago 943MB
node 10 01b816051d34 2 weeks ago 911MB
circleci/python 3.7.6 0d2975896c73 5 weeks ago 1.43GB
ariv3ra/infrastructure-as-code101 latest cb21a36a2973 8 weeks ago 929MB
python 3.7.6 879165535a54 2 months ago 919MB
circleci/rust 1-buster 218329b929cf 2 months ago 1.68GB
rust 1-buster f5fde092a1cd 2 months ago 1.19GB
python 3.7.4 9fa56d0addae 6 months ago 918MB
docker build -t <image name> .creates a Docker image with a tag using-t
Docker naming convention The above command, in conjunction with a valid Dockerfile, builds a Docker image based on the execution commands defined in the Dockerfile. One critical element to building images and starting containers is understanding the Docker naming convention. Building Docker images using this command specifies a name for the target Docker image. Docker images utilizes a naming convention made up of slash-separated name components that may contain lowercase letters, digits, and separators. A separator is defined as a period, one or two underscores, or one or more dashes. A name component may not start or end with a separator. A docker tag name must be valid ASCII and may contain lowercase and uppercase letters, digits, underscores, periods, and dashes. A tag name may not start with a period or a dash and may contain a maximum of 128 characters. You can group your images together using names and tags. In this post, we will use this naming convention.
Now that we’ve discussed naming conventions let’s build a Docker image based on the Dockerfile in our example above. In this post we can utilize the punkdata/nodejs-circleci git repo. Clone it locally and $ cd into the project directory.
Then run this command to build a new Docker image based on the project source code and the example Dockerfile:
docker build -t tutorial/circleci-node:10 .
After running this build command, you will see results similar to this:
Sending build context to Docker daemon 98.07MB
Step 1/7 : FROM node:10
---> 01b816051d34
Step 2/7 : WORKDIR /usr/src/app
---> Using cache
---> 12b2edc2b97c
Step 3/7 : COPY package*.json ./
---> Using cache
---> 53b5b8e4e654
Step 4/7 : RUN npm install --only=production
---> Using cache
---> eefdf560bc4d
Step 5/7 : COPY . .
---> aa7d54e955c6
Step 6/7 : EXPOSE 5000
---> Running in cc427dbafdcc
Removing intermediate container cc427dbafdcc
---> 4ce9084e39eb
Step 7/7 : CMD [ "npm", "start" ]
---> Running in f6e854599ddc
Removing intermediate container f6e854599ddc
---> 79a8d94cbf42
Successfully built 79a8d94cbf42
Successfully tagged tutorial/circleci-node:10
Run the docker images command and you will see your newly created Docker image listed in the results.
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
tutorial/circleci-node 10 79a8d94cbf42 4 minutes ago 1.03GB
Now you have an awesome Docker image built and ready to use. In the next section I’ll create new instances of this Docker image in the form of Docker containers
Docker containers
- Docker containers are designed to run applications in isolation and at scale. They allow for streamlining the management and implementation of applications.
Before we start creating and running Docker containers, I’ll add some context. Docker containers are objects that are based on and spawned from Docker images. Docker images are templates. I liken them to cookie cutter molds. Cookie cutter molds enable you to quickly and consistently produce individual cookies from dough. I liken these cookies to Docker containers which are distinguished in shape by the type of cookie mold, or Docker container, used in the cutting/building process. So using my cookie cutter analogy, Docker images are the cookie cutter and the individual cookies cut using that cutter are equivalent to Docker containers.
Now that I’ve probably made you a bit hungary for cookies, let’s start running some containers.
docker runruns a command in a new container.
This command is the most important command of the Docker runtime and is responsible for creating and starting Docker containers.
Let’s start a new container named nodetest01:
docker run -d -p 5000:5000 --name nodetest01 tutorial/circleci-node:10
Congrats! You should now have the Node.js image running in the new container that you just created available on port 5000. Open a web browser and go to this address: http://localhost:5000. It will take you the static “Welcome to CI/CD 101 using CircleCI!” web page served by this application. You can also verify your container is running by executing a docker ps command in the terminal. We’ll discuss this in the next section. Before that, let’s start an other instance of the application by running another container.
We started our first container with the name nodetest01 and since container names must be unique, we’re going to name our new container nodetest02. Another change that must be made in our new container is the port number. We’ll change it from 5000 to 5001 since applications cannot occupy the same port number on the same network interface.
Execute the following docker run command in the terminal:
docker run -d -p 5001:5000 --name nodetest02 tutorial/circleci-node:10
Awesome! You now have two containers running on your local machine. You can go to this address http://localhost:5001 in a browser and see the app running in your second Docker container.
The docker run command is very robust and has many property and configuration flags which I won’t be able to address or demonstrate in this post. I highly recommend that you read and familiarize yourself with the many ways that you can execute and run Docker containers. You can read up on Docker containers commands here.
docker pslists all of the running Docker containers. This command serves a similar purpose as thedocker imagecommand and lists actively running containers. Use the-aflag to show all of the running and not running containers.
Run this command in a terminal and you should be able to see both of the previously created containers running in the results:
docker ps
The results will be similar to this showing both the node01 and node02 containers still running.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dfd30181e15c tutorial/circleci-node:10 "docker-entrypoint.s…" 12 minutes ago Up 12 minutes 0.0.0.0:5001->5000/tcp nodetest02
0efaf7f11780 tutorial/circleci-node:10 "docker-entrypoint.s…" 28 minutes ago Up 28 minutes 0.0.0.0:5000->5000/tcp nodetest01
-
docker start <container name>starts an existing container. -
docker stop <container name>stops an existing container.
This command is only used to start existing containers that were created with the docker run command. Unless the --rm=true flag is specified with the docker run command, newly created containers will persist and can be reused with the docker start and docker stop commands.
Let’s stop some running containers by executing these commands:
docker stop nodetest01 nodetest02
These running containers should now be inactive and stopped. Since they’re stopped, run the docker ps -a command and you will see results similar to the following indicating stopped containers.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dfd30181e15c tutorial/circleci-node:10 "docker-entrypoint.s…" 31 minutes ago Exited (0) 38 seconds ago nodetest02
0efaf7f11780 tutorial/circleci-node:10 "docker-entrypoint.s…" About an hour ago Exited (0) 38 seconds ago nodetest01
Though these containers are stopped they will persist and can be restarted using the docker start command.
Lets start the nodetest01 container so we can learn how to use the docker logs features in the next section.
docker start nodetest01
Now run the docker ps -a command. Notice the status for this container reads similar to Up about a 30 seconds or something to that effect.
docker logsfetches a container’s logs.
This command enables developers to see logs that are present at the time of execution. Since your apps are running inside these containers, the logs command is useful for reading out critical app outputs. This helps in understanding how the app is performing and any debugging/troubleshooting that might be required.
Let’s run the docker logs command against the nodetest01 container:
docker logs nodetest01
After running this command you should see results similar to what’s shown below. It shows that the app inside container is up and running.
Node server is running..
> nodejs-circleci@0.0.1 start /usr/src/app
> node app.js
Node server is running..
docker rm <container name>deletes a container or multiple containers.
Persisting Docker containers consumes disk space and resources on its host. Operationally, it doesn’t make much sense to persist every container on disk since in most cases containers are disposable and should be used that way. So when a container is no longer needed it should be permanently discarded from the host. The docker rm command permanently deletes containers from the host. The nodetest02 container is no longer needed so let’s save some disk space and delete it. Note this command will only delete inactive/stopped containers and will error out if it’s run against an active/running container.
Run the docker rm command against the nodetest02 container:
docker rm nodetest02
You should no longer see the nodetest02 container on the host after running docker ps -a. This command has other properties and flags and you should familiarize yourself with them here.
docker rmideletes Docker images locally.
This command applies to deleting Docker images from a host. If there are existing containers on the host that are based on a specific image, then this command will not allow the deletion of the container based on the image. You must first ensure the container is stopped and deleted before you can delete Docker images using this docker rmi command. Read up on the docker rmi command here
Docker image deployments
Many Docker images are publicly available and hosted on the Docker Hub Registry which is an online central hosting solution for Docker images. Docker Hub enables anyone with an internet connection to pull down publicly available images from the registry to their local machines or servers. It also enables registered users to upload and publicly share any of their containers so that anyone can pull down that image from Docker Hub.
In the following sections I’ll briefly discuss some of the Docker Hub related commands.
docker pullpulls an image or a repository from a registry.
This command enables you to download a valid Docker image from Docker Hub. Publicly available Docker images do not require authentication. If the registry is private you will need to authenticate using an assigned credential usually in the form of a username and password.
docker loginlogs into a Docker registry.
This command enables you to authenticate against a Docker Registry. If you’re using this login command you should populate a .txt file with your Docker Hub account username and password to protect these credentials from exposure.
docker pushpushes an image or a repository to a registry.
This command enables users to upload images to a Docker registry and requires valid credentials.
Summary
So there you have it. In this post I went a bit deeper in explaining Docker terminology and commands as well as containers and their usage. The commands I discussed and demonstrated, are the most widely used commands in the Docker runtime and learning a bit more about them and their properties and flags will definitely help you level up your Docker and container skills.
Familiarizing yourself with these basic commands will remove one of the larger barriers that new users of CircleCI encounter.
If you want to learn more about CircleCI check out the documentation site, and if you really get stuck you can reach out to the CircleCI community via the https://discuss.circleci.com/ community/forum site.
