CircleCI Server v3.x Configuring External Services

External Database and Vault Setup for CircleCI Installations

This document describes how to configure the following external services for use with a CircleCI server 3.0 Installation:

  • PostgreSQL v12.6

PostgreSQL

Best Practices for your PostgreSQL

Consider running at least two or more PostgreSQL replicas to enable recovery from primary failure and for backups. Following are the recommended specifications of the PostgreSQL machines:

# of Daily Active Users # of PostgreSQL Replicas CPU RAM Disk NIC Speed

<50

2

8 Cores

16 GB

100 GB

1 Gbps

50 - 250

2

8 Cores

16 GB

200 GB

1 Gbps

250 - 1000

3

8 Cores

32 GB

500 GB

10 Gbps

1000 - 5000

3

8 Cores

32 GB

1 TB

10 Gbps

5000+

3

8 Cores

32 GB

1 TB

10 Gbps

Preparing the Database

If you are configuring an externalized PostgreSQL instance as part of a new CircleCI server 3.x install, it is recommended that you log in to your PostgreSQL instance and run the following commands. If you plan on using a different method, you must ensure that the databases and extensions outlined below are configured prior to running CircleCI server 3.x for the first time.

    CREATE DATABASE circle;
    \connect circle
    CREATE EXTENSION IF NOT EXISTS "pgcrypto";
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE contexts;
    \connect contexts
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE conductor_production;
    \connect conductor_production
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE contexts_service_production;
    \connect contexts_service_production
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE cron_service_production;
    \connect cron_service_production
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE domain;
    CREATE DATABASE permissions;
    \connect permissions
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE vms;
    \connect vms
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE orbs;
    \connect orbs
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE orb_analytics;
    \connect orb_analytics
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE builds_service;
    \connect builds_service
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
    CREATE DATABASE distributor;
    \connect distributor
    CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

Backing Up PostgreSQL

PostgreSQL provides official documentation for backing up and restoring your PostgreSQL 12 install which can be found here.

We strongly recommend the following:

  • Taking daily backups

  • Keeping at least 30 days of backup

  • Using encrypted storage for backups as databases might contain sensitive information

  • Performing a backup before each upgrade of CircleCI Server.



Help make this document better

This guide, as well as the rest of our docs, are open-source and available on GitHub. We welcome your contributions.