Note from the publisher: You have managed to find some of our old content and it may be outdated and/or incorrect. Try searching in our docs or on the blog for current information.

Please note: This post is specifically about our SaaS offering. If your team uses a VPN, our Enterprise product will be a better fit for you. Drop us a line to talk more.

One questions users often ask support about is how to create a VPN tunnel within their builds to access a network resource securely. There’s a few reasons why this doesn’t work on our platform. One is due to how VPNs create TUN/TAP devices, and the other has to do with unprivileged LXC containers. This post will address common concerns and questions about using CircleCI with VPNs.

TUN/TAP devices

What is a TUN/TAP device?

A TUN/TAP device is a specific device that is created and used for VPN connections. It acts as an Ethernet device, but instead of receiving data directly from the physical device, it receives data from a user space program (which would be whichever VPN type you’re using, like L2TP, PPP, Cisco, etc.). It then sends the data back to the user space program, which then sends them to the “device” encrypted to be transported to the destination.

What does this have to do with my builds?

VPN connection software creates that device using a specific device node (like /dev/net/tun), and registers it as tunX or tapX. The user space program that is creating the VPN connection needs to have the proper privileges to create the device. The drivers to use the device also need to be compiled into the kernel to be used correctly. While users do have sudo access to their builds, there is a specific reason this doesn’t work for your builds, and that’s where LXC comes into play.

Unprivileged LXC Containers

What is an LXC Container?

When a user runs a build, we create an LXC container. The linux kernel provides different “views” of the system to different running processes. This allows physical resources, like I/O, CPU, RAM, etc. to be “compartmentalized” logically. The kernel also allows the using of two main components: control groups and POSIX file capabilities. The control groups allow the kernel to group together one or more processes and then specify resources to be allocated to that control group. POSIX file capabilities simply allow the files being used by those processes to have more granular security than the standard “root” and “user” privilege segregation.

What is an unprivileged LXC container?

Unprivileged LXC containers are a more secure version of LXC containers, and are what we use at CircleCI. In a privileged LXC container, the root UID is the same (0) for both the host machine and the container. This means that if a user were, say through the /proc or /sys filesystem, or through some syscall, able to break out of the container, they’d be UID 0 (or root) on the host machine, and thus have root access to the host as well as every other container/resource that’s on that host machine.

Unprivileged LXC containers use user namespaces to assign users an unused range of user and group ID’s on the host machine (typically 100,000 through 165,000). This means that if a user were able to break out of the container, the containers UID of 0 would map to UID 100,000 on the host machine. This would essentially grant them the same privileges as a nobody user, and greatly restrict what, if anything, that they’d be able to do on the host.

Why does this mean I can’t create a VPN?

When the “root” user within the container attempts to make a TUN/TAP device, it needs more privileges on the host than the limited access it does have. This restricts multiple operations, including but not limited to:

  • Mounting certain filesystems
  • Creating device nodes (like the ones needed to create the TUN/TAP device)
  • Or any operation that requires privileges outside of the ones assigned to the UID/GID range that - LXC allocates to containers (that 100,000 - 165,000 range).

Without being able to create the TUN/TAP device on the unprivileged container, any user on any platform using unprivileged LXC containers would not be able to use VPN services. This means that as long as we’re using unprivileged LXC containers, creation of TUN/TAP devices won’t be supported, unless a change gets made upstream with LXC to allow a way for us to be able to allow users to create those devices.

Does this mean there’s nothing you can do? No, of course not. There are some other options available. We’ll publish a follow-up post outlining these, as well as showing how they can be accomplished, in the future.