Orb Authoring Process

Introduction

This orb authoring guide assumes you have read the Introduction to authoring an orb document and claimed your namespace. At this point you are ready to develop an orb.

Whether you are writing your first orb or getting ready for production level, we recommend using our orb development kit to get started. Alternatively, as orbs are packages of reusable configuration, they can be written manually, as singular yaml files, and published using our circleci orb cli.

Orb Development Kit

The orb development kit refers to a suite of tools that work together to simplify the orb development process, with automatic testing and deployment on CircleCI. The orb development kit is made up of the following components:

Getting started

To begin creating your new orb with the orb development kit, follow these steps. The starting point is creating a new repository on GitHub.com.

Ensure the organization on GitHub is the owner for the namespace for which you are developing your orb. If this is your own personal organization and namespace, you need not worry.

  1. Create a new GitHub repository.
    The name of your repository is not critical, but we recommend something similar to “myProject-orb”. Orb Registry

    When complete, you will be brought to a page confirming your new repository and you should see the generated git URL. Note down the git URL, you will need it in step 4. You can select SSH or HTTPS, which ever you can authenticate with. Orb Registry

  2. Open a terminal and initialize your new orb project using the orb init CLI command.
    circleci orb init /path/to/myProject-orb
    

    The circleci orb init command is called, followed by a path that will be created and initialized for our orb project. It is best practice to use the same name for this directory and the git project repo.

  3. Choose the fully automated orb setup option.
    ? Would you like to perform an automated setup of this orb?:
      ▸ Yes, walk me through the process.
     No, I'll handle everything myself.
    

    When choosing the fully automated option, the Orb-Project-Template will be downloaded and automatically modified with your customized settings. The project will be followed on CircleCI with an automated CI/CD pipeline included.

    For more information on the included CI/CD pipeline, see the Orb Publishing Process documentation. Alternatively, if you would simply like a convenient way of downloading the Orb-Project-Template you can opt to handle everything yourself.

  4. Answer questions to configure and set up your orb.
    In the background, the orb init command will be copying and customizing the Orb Project Template based on your inputs. There are detailed README.md files within each directory that contain helpful information specific to the contents of each directory. You will also be asked for the remote git repository URL that you obtained back in step 1.

    The Orb Project Template contains a full CI/CD pipeline (described in Orb Publishing Process) which automatically packs, tests, and publishes your orb.

    In the setup process you will be asked if you would like to save your Personal API Token into an orb-publishing context. Saving this token is necessary for publishing development and production versions of your orb.

    Ensure the context is restricted
    Restrict a context by navigating to Organization Settings > Contexts.

    After completing your orb, you should see a new context called orb-publishing. Click into orb-publishing and add a Security Group. Use Security Groups to limit access to users that are allowed to trigger jobs. Only these users will have access to the private Personal API Token.

    For more information, see the Contexts guide.

  5. Push the changes up to Github.
    During the setup process, the orb init command takes steps to prepare your automated orb development pipeline. The modified template code produced by the CLI must be pushed to the repository before the CLI can continue and automatically follow your project on circleci.com. Run the following command from a separate terminal when prompted to do so, substituting the name of your default branch:
    git push origin <default-branch>
    

    Once complete, return to your terminal and confirm the changes have been pushed.

  6. Complete and write your orb.
    The CLI will finish by automatically following your new orb project on CircleCI and generating the first development version <namespace>/<orb>@dev:alpha for testing (a hello-world sample).

    You will be provided with a link to the project building on CircleCI where you can view the validation, packing, testing, and publication process. You should also see the CLI has automatically migrated you into a new development branch, named alpha.

    From your new branch you are now ready to make and push changes. From this point on, on every commit, your orb will be packed, validated, tested (optional), and can be published.

    When you are ready to deploy the first major version of your orb, find information on deploying changes with semver versioning in the Orb Publishing Process guide.

Writing your orb

Before you begin working on your orb, ensure you are on a non-default branch. We typically recommend starting your orb on the alpha branch.

$ git branch

* alpha
  main

If you have run the circleci orb init command, you will automatically be in the alpha branch and have a repository with .circleci and src directories.

Example: Orb Project Structure

type name
.circleci
.github
src
.gitignore
CHANGELOG.md
LICENSE
README.md

Orb source

Navigate to the src directory to look at the included sections.

Example: Orb Project “src” Directory

type name
commands
examples
executors
jobs
@orb.yml

The directories listed above represent orb components that can be included with your orb. @orb.yml acts as the root of your orb. You may additionally see scripts and tests directories in your project for optional orb development enhancements, which we will cover in the Scripts section and the Orb Testing Methodologies guide.

Each directory within src corresponds with a reusable configuration component type, which can be added or removed from the orb. If, for example, your orb does not require any executors or jobs, these directories can be deleted.

@orb.yml

@orb.yml acts as the “root” to your orb project and contains the config version, the orb description, the display key, and imports any additional orbs if needed.

Use the display key to add clickable links to the orb registry for both your home_url (the home of the product or service), and source_url (the git repository URL).

version: 2.1

description: >
  Sample orb description

display:
  home_url: "https://www.website.com/docs"
  source_url: "https://www.github.com/EXAMPLE_ORG/EXAMPLE_PROJECT"
Commands

Author and add Reusable Commands to the src/commands directory. Each YAML file within this directory will be treated as an orb command, with a name which matches its filename.

Below is the greet.yml command example from the Orb Project Template.

description: >
  # What will this command do?
  # Descriptions should be short, simple, and clear.
parameters:
  greeting:
    type: string
    default: "Hello"
    description: "Select a proper greeting"
steps:
  - run:
      name: Hello World
      command: echo << parameters.greeting >> world
Examples

Author and add Usage Examples to the src/examples directory. Usage Examples are not for use directly by end users in their project configs, but they provide a way for you, the orb developer, to share use-case specific examples on the Orb Registry for users to reference.

Each YAML file within this directory will be treated as an orb usage example, with a name which matches its filename.

View a full example from the Orb Project Template.

Executors

Author and add Parameterized Executors to the src/executors directory.

Each YAML file within this directory will be treated as an orb executor, with a name that matches its filename.

View a full example from the Orb Project Template.

Jobs

Author and add Parameterized Jobs to the src/jobs directory.

Each YAML file within this directory will be treated as an orb job, with a name that matches its filename.

Jobs can include orb commands and other steps to fully automate tasks with minimal user configuration.

View the hello.yml job example from the Orb Project Template.

description: >
  # What will this job do?
  # Descriptions should be short, simple, and clear.

docker:
  - image: cimg/base:stable
parameters:
  greeting:
    type: string
    default: "Hello"
    description: "Select a proper greeting"
steps:
  - greet:
      greeting: "<< parameters.greeting >>"

Scripts

One of the major benefits of the orb development kit is a script inclusion feature. When using the circleci orb pack command (automated when using the orb development kit), you can use the value <<include(file)>> within your orb config code, for any key, to include the file contents directly in the orb.

This is especially useful when writing complex orb commands, which might contain a lot of bash code, (although you could use python too!).

parameters:
  to:
    type: string
    default: "World"
    description: "Hello to whom?"
steps:
  - run:
      environment:
        PARAM_TO: <<parameters.to>>
      name: Hello Greeting
      command: <<include(scripts/greet.sh)>>
parameters:
  to:
    type: string
    default: "World"
    description: "Hello to whom?"
steps:
  - run:
      name: Hello Greeting
      command: echo "Hello <<parameters.to>>"
Why include scripts?

CircleCI configuration is written in YAML. Logical code such as bash can be encapsulated and executed on CircleCI through YAML, but, for developers, it is not convenient to write and test programmatic code within a non-executable format. Also, parameters can become cumbersome in more complex scripts as the <<parameter>> syntax is a CircleCI native YAML enhancement, and not something that can be interpreted and executed locally.

Using the orb development kit and the <<include(file)>> syntax, you can import existing scripts into your orb, locally execute and test your orb scripts, and even utilize true testing frameworks for your code.

Using parameters with scripts

To keep your scripts portable and locally executable, it is best practice to expect a set of environment variables within your scripts and set them at the config level. The greet.sh file, which was included with the special <<include(file)>> syntax above in our greet.yml command file, looks like this:

echo Hello "${PARAM_TO}"

This way, you can both mock and test your scripts locally.

Testing orbs

Much like any other software, there are multiple ways to test your code and it is entirely up to you as the developer to implement as many tests as desired. Within your config file right now there will be a job named integration-test-1 that will need to be updated to test your orb components. This is a type of integration testing. Unit testing with orbs is possible as well.

Read our full Orb Testing Methodologies documentation.

Publishing your orb

With the orb development kit, a fully automated CI and CD pipeline is automatically configured within .circleci/config.yml. This configuration makes it simple to automatically deploy semantically versioned releases of your orbs.

For more information, see the Orb Publishing Process guide.

Categorizing your orb

You can categorize your orb for better discoverability in the Orb Registry. Categorized orbs are searchable by category in the Orb Registry. CircleCI may, from time to time, create or edit orb categorizations to improve orb discoverability.

Listing categories

You can select up to two categories for your orb. These are the available categories:

  • Artifacts/Registry
  • Build
  • Cloud Platform
  • Code Analysis
  • Collaboration
  • Containers
  • Deployment
  • Infra Automation
  • Kubernetes
  • Language/Framework
  • Monitoring
  • Notifications
  • Reporting
  • Security
  • Testing

The list of categories can also be obtained by running the circleci orb list-categories CLI command. You can view the detailed docs for this command here.

Add an orb to a category

Add your orb to your chosen category by running circleci orb add-to-category <namespace>/<orb> "<category-name>". You can view the detailed docs for this command here.

Remove an orb from a category

Remove an orb from a category by running circleci orb remove-from-category <namespace>/<orb> "<category-name>". You can view the detailed docs for this command here.

Viewing an orb’s categorizations

To see which categorizations have been applied an orb, check the output of circleci orb info <namespace>/<orb> for a list. You can view the detailed docs for this command here.