Over the last year we have had over 5,000 candidates pass through our engineering hiring process.

That’s 5,000 recruiter screens, almost a thousand interviews, 177 final interviews, and 63 job offers.

And it’s 5,000 chances to ask ourselves how we can continue to improve our hiring process.

Since we last wrote about how we interview engineers at CircleCI over two years ago, we raised two more funding rounds and almost tripled the size of our engineering department. To keep up with our growing organization, we also needed to update our hiring process to a model that fits our current needs better.

This post will cover new structures put in place, the reasons behind those structures, and a detailed rundown of the current interview process we use to hire engineers at CircleCI.

What hasn’t changed

We still don’t believe in whiteboarding interviews, or other interview practices that are still common in our industry and mostly designed to set candidates up for failure. Our main goals for interviewing candidates are:

  1. Identify the best candidates and invite them to join our company.
  2. Provide all candidates with a positive experience.

To achieve these goals, we rely on consistency and structure. We use a structured interview process with standardized questions. We also design all our roles and job descriptions and evaluate candidates based on our Engineering Competency Matrix.

Last year, we introduced interviewer training for all our interviewers. Our interviewer training is made up of a self-guided training and interview shadowing. The self-guided training covers areas such as our hiring process and interviewers’ roles in it, cognitive biases and how to mitigate them, the legal side of interviewing, an introduction to our Applicant-Tracking System (ATS), and how to prepare, run, and close out an interview, as well as how to evaluate candidates. We’ve also used mock interview sessions when interviewers have needed additional training for specific interviews. So far, the responses have been very positive, and everyone having the same understanding of how to run interviews effectively has also helped us level up as an organization.

Continuous learning

One of our company values is “quick and continuous”:

We value quickly iterating and shipping, then refining and adjusting, versus trying to do everything at once. We opt for quick experimentation, getting things over the line and continuously learning.

Continuously learning and evolving is at the core of what we do: it’s the heart of our product, as well as of how we operate as an organization. We continuously invest in improving both the product we build, as well as how we work as a team. Improving how we hire is an important part of that.

Responding to candidate feedback

We ask every candidate who interviews with us about feedback on their experience. Over the years, many people have shared their thoughts with us. We look at each feedback survey response that we receive, and have tweaked our hiring process with our teams to address those feedback points.

They’ve pointed out areas that we’re doing well in: we have many great people on the teams who are excited to talk about what they do and answer candidates’ questions; we’ve also heard that candidates enjoyed the different angles our interviews focused on, as well as the interactions with our recruiting team.

We have also heard about areas that we could improve in: making sure to maintain a close communication loop with candidates, as well as clarifying roles and responsibilities early on in the process.

One recurring piece of feedback was the length of our process: candidates perceived it as thorough, but that it felt quite long. We’d also lost good candidates in the process because they received offers elsewhere before we’d completed our interview process. We knew we needed to create a more streamlined process that could still work for a globally distributed team with many time zones, and maintain our standards for thorough interviewing. Here is the process we use now.

Updated Interview Structure

Interview structure for Engineering roles up to Senior Level

Our interview process has been fully remote since March 2019, and interviews usually take place via video conference software. As of August 2020, our interview structure consists of the following stages:

1. Recruiter meeting: Our technical recruiter meets with the candidate and discusses the position they applied for, their career interests, and what they’re looking for in their next role. We’ve heard from many candidates who enjoyed their interactions with our recruiting team tremendously – they’re the ones who help make this a smooth experience for our candidates, and work very hard every day to get to know candidates well and find the best person for the job.

2. Hiring Manager screen: The hiring manager for the position talks with the candidate to assess experience relevant to the role, communication and collaboration skills as well as other core skills from our competency matrix, and ability to handle duties specific to the role.

3. Technical interview: We now partner with a company that specialises in technical interviewing to conduct the technical assessment for all candidates and all roles. This interview replaces a take-home exercise which we used in the past, and allows us more standardized evaluations. It also offers great flexibility to candidates in interview scheduling (including interviews at night and on weekends). The goal of the technical interview is to objectively determine the candidate’s level of technical skills. Prior to the interview, all candidates get access to a comprehensive prep environment for their interview, which includes an intro video, logistical details and an A/V test, FAQ, best practices, and a sample coding environment. The interview itself starts with a short intro and a knowledge discussion based on question sets that we select for each role individually, followed by a coding challenge. Each technical interview is then reviewed by a member of our team for alignment and consistency.

4. Team interview: This stage is a conversation with two members of the team that the candidate applied to join. We have three goals for this meeting. We want to understand what technical skills a candidate brings. We want to asses skills in communication, collaboration, and user value delivery. Finally, we want to determine whether we think they could succeed in this role. We’re always striving to expand the amount of perspectives, backgrounds, and experiences we have represented on our team.

5. Wrap-up conversation: Each interview includes time for candidates to ask us questions, and sometimes there are questions left after the last interview, so, upon request, we offer an optional wrap-up conversation with the hiring manager or the senior engineering manager of the domain in which the role would be. For some roles, the wrap-up conversation is part of the hiring process by default.

Interview flow for Staff level roles and above

For some of our more senior roles (Staff level and above in our engineering competency matrix), we still rely on the process that we previously shared, and have replaced the “micro-skills take-home problem” replaced with an interview with our partner for technical assessments. This allows us and our candidates to have more in-depth conversations that we believe are important for roles at these levels.

If you have any questions about the interview process or stages, feel free to ask the hiring manager or recruiter for the role that you’re applying for. We understand that interviewing can be a stressful experience, and want to do our best to make you feel at ease during this process.

Providing a positive candidate experience

A big part of the interviewer training that all our interviewers go through is how to provide a positive candidate experience: We want to provide every candidate with a great experience, and hope to leave a lasting positive impression of our company and interview process – regardless of whether we make them an offer or not. All our trusted interviewers play an important role in providing this experience to our candidates. Interviewing is highly stressful, and some companies put candidates through trials and tests that only makes it worse. (whiteboard interviews seem to still be around, apparently). We want to be an example of how things can be better. While an interviewer may be speaking with a lot of people, for many applicants this is their only time speaking with us, and we all want to leave a positive impression.

Specifically, we ask our interviewers the following:

  • Always be prepared (we have detailed instructions for this) and always show up on time
  • If you’re not sure about how to pronounce the candidate’s name, ask (and leave a note for other interviewers to see).
  • Listen well, don’t interrupt. Remember that this is about giving them a chance to share how awesome they are; keep the conversation focused on the candidate and their experiences.
  • Connecting with candidates can be difficult remotely, and the remoteness can exacerbate differences in personalities. Please be aware that just because you’re not “clicking” with a candidate during your conversation with them, this doesn’t mean they’re not qualified.
  • In case of technical difficulties or other issues, be patient.

We appreciate every candidate deciding to interview with us, and want to make sure this appreciation comes across in our practices.

Continuing to improve our practices

We will continue improving this process as we use it more and hear feedback from our interviewer teams internally, as well as from candidates. Several candidates have moved through our modified process in the last weeks already and have shared their feedback with us – thank you! Your feedback has helped us iterate on this further, and we appreciate your taking the time to help us improve. If you’ve been through our hiring process and have feedback, please share it with us in the candidate survey, we’d love to hear from you.

If you’re interested in joining our team, find more about us and our open positions on our jobs page.