Infrastructure as code (IaC) is the process of managing and provisioning cloud and IT resources via machine readable definition files and is a part of modern continuous integration pipelines. IaC enables organizations to provision, manage, and destroy compute resources using modern DevOps tools. IaC enables this by statically defining and declaring these resources in code, then deploying and dynamically maintaining these resources via code.
In recent interactions with developers, I’ve realized that many are either not aware of IaC, or are aware but have abandoned their initial efforts because ramping up was too cumbersome. In this post, I will discuss the benefits of IaC and some of the tooling option currently available along with a brief demonstration on how to use them in hopes of easing the transition to using IaC. I will focus on the two most popular IaC tools used in the industry today: HashiCorp’s Terraform and Pulumi.
Prerequisites
Before you get started, you’ll need to have these things:
- A Google Cloud account
- Create a Google Cloud project
- Create a Pulumi API token and save it
- A local install of the Google Cloud SDK CLI
- A local install of the Docker client
Create Google Cloud project credentials
You will need to create Google Cloud credentials in order to perform administrative actions using IaC tooling.
- Go to the Create Service Account Key page
- Select the default service account or create a new one, select JSON as the key type, and click Create
- Save this JSON file in the
~/.config/gcloud/directory and rename the file tocicd_demo_gcp_creds.json. This is very important for enabling the Google Cloud CLI in the container later on
Create a Pulumi API token
You will need to get a Pulumi API token to store your project state on Pulumi’s backend cloud.
- Go to app.pulumi.com/
- Click Select an Organization on the upper left side of the page and select your account
- Click Settings
- Click Access Tokens on the left side of the page
- Click New Access Token on the right side of the page
- Click Create and save the new API token that was created. This will be used to initialize your Pulumi project later
Docker pull IaC101 Docker image
At this point, all of the prerequisites are completed and you’re ready to pull the docker image for this workshop. In terminal type the following command:
docker pull ariv3ra/iac101
You should now have the ariv3ra/iac101 docker image locally. You can verify buy running docker images and you should see the image listed in the results.
Docker run IaC101 container
Now we’ll create a new Docker container based on the ariv3ra/iac101 image which has all the IaC tools and code pre-baked.
Mounting the /.config/gcloud/ directory
Before running the new container you need the absolute path to your ~/.config/gcloud/ directory. On my MacOS machine, my absolute path is /Users/angel/.config/gcloud/. Be sure to get your absolute file path for the gcloud directory on your local machine.
Run IaC101 container with mounts
Run this command in a terminal run but be sure to replace the <your absolute path here> bit with your actual absolute path to your local glocud/directory:
docker run -it --name iactest --mount type=bind,source=<your absolute path here>.config/gcloud/,target=/root/.config/gcloud/ ariv3ra/iac101
IaC101 container is running
After running the previous docker run command, your IaC101 container will be up and running and your terminal shell has dropped you into the running container. Every command you run in your terminal shell will be executed in the Docker container from now on until you manually exit the container. The container has the Terraform and Pulumi CLI tools installed, as well as example code for each that creates infrastructure in their respective tools. The container projects files are mapped as follows.
projects/
|_ terraform/gcp/compute/ # Contains the Terraform code files
|_ pulumi/gcp/compute/ # Contains the Pulumi code files
HashiCorp Terraform
HashiCorp Terraform is a an open source tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing service providers as well as custom in-house solutions.
Configuration files describe the components needed to run a single application or your entire datacenter on Terraform. Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create incremental execution plans which can be applied.
The infrastructure Terraform can manage includes low-level components such as compute instances, storage, and networking, as well as high-level components such as DNS entries, SaaS features, etc.
Provisioning infrastructure with Terraform
Let’s start with provisioning some resources in GCP using Terraform code. The main.tf in the terraform/gcp/compute/ is the code that has our infrastructure defined. In this file we’re creating a new compute instance that will install and run a Python Flask app packaged in a Docker container. The Terraform code will also create some firewall rules that will allow public access to the app over port 5000.
To provision this, run this command in the terminal:
cd ~/project/terraform/gcp/compute/
Initializing Terraform
While in ~/project/terraform/gcp/compute/ run this command:
terrform init
You will see results similar to the results below.
root@d9ce721293e2:~/project/terraform/gcp/compute# terraform init
Initializing the backend...
Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "google" (hashicorp/google) 3.10.0...
* provider.google: version = "~> 3.10"
Terraform has been successfully initialized!
Previewing with Terraform
Terraform has a command that allows you to dry run and validate your Terraform code without actually executing anything. The command is called terraform plan which also graphs all the actions and changes that Terraform will execute against your existing infrastructure. In the terminal run:
terraform plan
You will see results similar to this.
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# google_compute_firewall.http-5000 will be created
+ resource "google_compute_firewall" "http-5000" {
+ creation_timestamp = (known after apply)
+ destination_ranges = (known after apply)
}
# google_compute_instance.default will be created
+ resource "google_compute_instance" "default" {
+ can_ip_forward = false
+ cpu_platform = (known after apply)
+ deletion_protection = false
+ guest_accelerator = (known after apply)
+ id = (known after apply)
+ instance_id = (known after apply)
+ label_fingerprint = (known after apply)
+ labels = {
+ "container-vm" = "cos-stable-69-10895-62-0"
}
+ machine_type = "g1-small"
}
Plan: 2 to add, 0 to change, 0 to destroy.
As you can see, Terraform is going to create new GCP resources for you based on the code in the main.tf file.
Terraform apply
Now you’re ready to create the new infrastructure and deploy the application. Run this command in the terminal:
terraform apply
Terraform will prompt you to confirm your command. Type yes and hit enter.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
Terraform will build your infrastructure and you will have an application up and running on GCP shortly after it completes. Please note that it will take 3-5 minutes for the application to come online after Terraform completes. It’s not an instant process because the backend systems are provisioning and bringing things online.
Once done, you will see results similar to this.
Outputs:
Public_IP_Address = 34.74.75.17
The Public_IP_Address value is the IP Address that your app is running on, so if you go to a browser and plug that number in plus the :5000 port address (it would look like this 34.74.75.17:5000) you will see your app running in GCP.
Terraform destroy
Now that you have proof that your Google Compute instance and your application work, run the terraform destroy command to destroy the assets that you created in this tutorial. You can leave it up and running, but be aware that there is a cost associated with any assets running on the Google Cloud Platform and you will be liable for those costs. Google gives a generous $300 credit for its free trial sign-up, but you could easily eat through that if you leave assets running. It’s up to you, but running terraform destroy will terminate any running assets.
Next, I’ll demonstrate how to perform infrastructure provisioning using Pulumi.
Pulumi SDK
Pulumi SDK is an open source framework for defining and deploying cloud apps and IaC. Pulumi enables developers to write code in their favorite language such as TypeScript, JavaScript, Python, and Go. This enables modern approaches to cloud applications and infrastructure without needing to learn another YAML or DSL dialect. This unlocks abstractions and reuse, as well as access to your favorite IDEs, refactoring, and testing tools. Master one toolchain and set of frameworks and go to any cloud (AWS, Azure, GCP, or Kubernetes) with ease.
Provisioning infrastructure with Pulumi
You’ve experienced building infrastructure using Terraform, now you’ll learn how to provision and deploy new infrastructure using Pulumi. The Pulumi example will create the same GCP resources previously using Pulumi code and tools. Pulumi enables you to define your infrastructure in programming languages which provides lots of flexibility. In this example, we’re using Python as our language. The __main.py__ file in the ~/project/pulumi/gcp/compute/ directory is where were going to define our infrastructure.
Be sure to have your Pulumi API token available. You’ll need it in the following sections.
The Pulumi example can be found in the ~/project/pulumi/gcp/compute/ directory. Run this command to change into the Pulumi example directory:
cd ~/project/pulumi/gcp/compute/
Installing dependiencies for Pulumi
Since we’re defining our IaC specs using Python we’ll need to first install the Pulumi Python SDK dependencies. In the terminal run this command:
pip install -r ../requirements.txt
Previewing with Pulumi
Pulumi has a dry-run command called pulumi preview. This command displays a preview of the updates to an existing stack whose state is represented by an existing state file. The new desired state is computed by running a Pulumi program, and extracting all resource allocations from its resulting object graph. These allocations are then compared against the existing state to determine what operations must take place to achieve the desired state. No changes to the stack will actually take place.
Run this command in the terminal:
pulumi preview
After running this you will see the following results prompting you for the Pulumi API token you created earlier. Paste the token into the terminal. After you paste the token hit enter.
Manage your Pulumi stacks by logging in.
Run `pulumi login --help` for alternative login options.
Enter your access token from https://app.pulumi.com/account/tokens
or hit <ENTER> to log in using your browser :
Note: When pasting the API token into the terminal, no values will be displayed. It will remain invisible for security purposes.
You may be prompted to select a stack in the terminal. If so, select the dev option then hit enter. You will see results similar to this.
Previewing update (dev):
Type Name Plan
+ pulumi:pulumi:Stack compute-dev create
+ ├─ gcp:compute:Network network create
+ ├─ gcp:compute:Address workshops-infra-as-code101 create
+ ├─ gcp:compute:Instance workshops-infra-as-code101 create
+ └─ gcp:compute:Firewall firewall create
Resources:
+ 5 to create
You’ve just enabled access to the Pulumi backend cloud which keeps track of the state of your infrastructures. Now you’re ready to run some code.
Pulumi up
To execute the Pulumi code, use the command pulumi up. This command creates or updates resources in a stack. The new desired goal state for the target stack is computed by running the current Pulumi program and observing all resource allocations to produce a resource graph. This goal state is then compared against the existing state to determine what create, read, update, and/or delete operations must take place to achieve the desired goal state in the most minimally disruptive way. Afterward, this command records a full transactional snapshot of the stack’s new state so that the stack may be updated incrementally again later on.
In the terminal run:
pulumi up
Select the yes option and hit enter. The Pulumi app will execute, and shortly after you’ll have a complete server running an application on GCP. You will see results similar to this.
Updating (dev):
Type Name Status
+ pulumi:pulumi:Stack compute-dev created
+ ├─ gcp:compute:Network network created
+ ├─ gcp:compute:Address workshops-infra-as-code101 created
+ ├─ gcp:compute:Firewall firewall created
+ └─ gcp:compute:Instance workshops-infra-as-code101 created
Outputs:
external_ip : "34.74.75.17"
instance_meta_data: {
gce-container-declaration: "spec:\n containers:\n - name: workshops-infra-as-code101\n image: ariv3ra/workshops-infra-as-code101:latest\n stdin: false\n tty: false\n restartPolicy: Always\n"
}
instance_name : "workshops-infra-as-code101"
instance_network : [
[0]: {
accessConfigs : [
[0]: {
natIp : "34.74.75.17"
network_tier : "PREMIUM"
}
]
name : "nic0"
}
]
Resources:
+ 5 created
Duration: 58s
In these results, you’ll see a external_ip key. It’s value is the IP Address to the public facing application exposed on port 5000. Just like the previous Terraform example, you can access the application in a browser. Remember to allow for a few minutes so the backend system can bring everything online.
Pulumi destroy
Pulumi has a command to terminate all of the cloud resources called pulumi destroy. This command deletes an entire existing stack by name. The current state is loaded from the associated state file in the workspace. After running to completion, all of this stack’s resources and associated state will be gone.
Run pulumi destroy in the terminal and select the yes option when prompted to permanently destroy the GCP resources created.
Summary
Congratulations! You’ve just leveled up and now have experience provisioning and deploying applications to GCP using the modern IaC tools Terraform and Pulumi. The following resources will help you expand your knowledge from here:
