OrbsFeb 26, 20197 min read

Creating automated build, test, and deploy workflows for orbs, part 1

Community & Partner Engineer, CircleCI

Fifteen weeks out from the launch of CircleCI orbs, there are more than 405 orbs in the Orb Registry. That’s upwards of 25 new orbs per week!

Amazingly, most of these orbs have been contributed by you, our amazing user community, or else by the folks in our new Technology Partner Program. However, there are nearly 30 orbs that have been published by CircleCI, and as CircleCI’s first Community & Partner Engineer, I’ve been spending the last few months working to develop a sustainable development pipeline for these orbs–a pipeline that will allow for building, testing, and deploying with as little manual intervention as possible.

This has been no small feat, as every orb is different. Some install and configure third-party tools and services. Some allow you to deploy to various cloud providers or test your code in external environments. And, some are simply syntactic sugar over existing CircleCI configuration features, making it easier for folks to use our platform.

What kinds of continuous integration and continuous deployment tooling can we use to support the development of all of these different types of orbs? More importantly, how can you make use of this tooling in your own orb development? At the end of the day, no one wants to be sitting at their computer typing git pull and circleci orb publish every time someone merges a pull request to an orb repository.

If you’re reading this, I’m going to assume you already know a bit about orbs. Perhaps you’ve published one, or perhaps you’ve started using them in your CircleCI projects. If not, I encourage you to check out some of our previous orbs blog posts! Support Engineer Kyle Tryon has some fantastic tips for writing your first orb; he also published an overview of his experiences creating CircleCI’s Slack orb (GitHub link), by far our most popular orb. For those looking for a more basic primer, the Creating Orbs and Reusing Config documentation is super helpful.

In this post, I’ll talk about setting up the ideal automated CI/CD process for a new orb. In my next post, I’ll do a full walkthrough of the Azure CLI orb and the automation that we built into it.

Now, let’s jump into the automated process for our new orb. What kinds of goals might such a process have?

Version control

Nestled firmly into the heart of the DevOps ecosystem, CircleCI is a core believer in the concept of “infrastructure as code.” Orbs are infrastructure, represented as code, so they should be built, tested, and deployed as would any other piece of software. To do that, your orb needs to live in a git repository.

Now, some orbs are very small. Your orb might only do one, simple thing. Others are much larger (check out our AWS ECS orb). For smaller orbs, especially those that are less beholden to external services, some kind of orbs monorepo might be appropriate; there’s actually a lot that can be tested, even in a repository with multiple orbs. You can see an example of this approach in our circleci-orbs repository, which is home to some of the first CircleCI-published orbs.

Over time, we quickly found that for many, more complex orbs, for example, those focused on third-party deployments, more granularity was needed. In these cases, a 1:1-orb:repository relationship is ideal. Remember, all of our CI/CD logic will have to live in our config.yml file, and trying to manage integration testing and automated publishing for multiple large orbs, within a single configuration file, would quickly get overwhelming.

Multiple levels of testing

Just as you would test a website or application in multiple ways, at multiple points in its development pipeline, we should do the same with orbs. Within the orbs ecosystem, what are these different levels of testing?

To paraphrase the original orbs SDK preview:

Schema Validation: This can be done with a single CLI command and it checks to see if the orb is written in well-formed YAML and conforms to the orb schema.
Runtime Testing: This requires setting up separate tests and running them inside a CircleCI build.
Integration Testing: This is likely the only testing needed for fairly advanced orbs or for orbs designed specifically as public, stable interfaces to 3rd-party services. Doing orb integration tests requires a custom build and your own external testing environments.

There’s also the concept of Expansion Testing: since orbs are essentially abstracted-out snippets of CircleCI configuration, we can test whether a given orb generates the CircleCI configuration syntax that we expect. In practice, I’ve found the cost-to-benefit ratio to be a bit low, as one needs to maintain not only the orb source itself, but also its corresponding desired expanded configuration syntax, and then manually keep the two in-sync in order for expansion tests to remain accurate. For some orbs, though, it may be helpful. Eddie Webb, CircleCI’s Director of Solutions, posted a great explanation of this approach (with examples!) in an orb-testing discussion on CircleCI Discuss.

Automated deployment

To me, this is the entire end goal of automating your orb development process. I couldn’t possibly maintain CircleCI’s growing fleet of orbs without this kind of automation!

As a new orb author, it may be tempting to simply publish manually at first. After all, we’re only talking about a few simple shell commands. Let me caution you against giving into this temptation. Internally, we’ve already fielded multiple questions from orb authors wondering why, after merging a pull request to their orb repository, they don’t see their updated orb in the registry, only to realize that they never actually automated the deployment portion of their orb testing and publishing process. And believe me, because I know from experience, it’s much smoother to automate from the start, than to go back and add it later.

There are a couple of nuances to automated orb deployment. The CircleCI CLI provides three different ways of publishing a production orb:

You can publish an orb.yml file directly to the registry, via circleci orb publish
You can promote a previously published dev version of an orb to a semantically versioned production orb, via circleci orb publish promote
You can increment an existing production orb version, e.g., from foo/bar@1.1.0 to foo/bar@1.1.1 (patch release), or 1.2.0 (minor), or 2.0.0 (major)

In practice, I’ve found that promoting dev orbs to production orbs to be the most useful automated deployment mechanism. It allows you to, on every commit, do basic testing/validation of orb source code, then conditionally publish a dev version of the orb, then run more expansive testing on the dev orb, and finally, conditionally promote your just-published dev orb to a production release.

Putting it all together

So, what does the config.yml look like for an orb built, tested, and deployed in this manner? To start, it’s not that complicated! We’ve abstracted out most of the basic mechanics of automated orb development into, well, an orb. To see a functional example, check out our Azure CLI orb, published just last month, as it is small and simple enough to be easily understood, yet also a bit complex, as it does integrate with Microsoft Azure, a third-party cloud provider. This orb’s config.yml contains two distinct workflows: one for basic validation and dev publishing, and another for integration/usage testing and possible orb deployment to production. In a follow up post, I will do a comprehensive walkthrough of the Azure CLI orb and these two workflows.

Orbs are the centerpiece of CircleCI’s growing open-source ecosystem, and as such, they’re better when you get involved. Tag your orbs on GitHub or Bitbucket with our circleci-orbs topic tag, so others can discover them more easily, and please post any questions or comments in the orbs section of CircleCI Discuss.