Note from the publisher: as of 9/2020, our latest information about our engineering hiring process can be found here.
Since the beginning of 2018, we have had over a thousand candidates pass through the engineering hiring process. Our engineering interview consists of four stages following an initial screen. We broke down the percentages of candidates that pass through each stage of our hiring process, and it looks like this.
Out of any group of 1000 applicants:
250 will pass the initial screen.
117 will pass the hiring manager phone screen.
44 will pass the micro-skills assessment portion of the interview.
7 will pass the macro-skills assessment.
Fewer than 3 will pass the on-site interview and receive offers.
Following our Series C, we’ve ramped up our hiring further, building on what we’ve learned so far. We’ve been talking a lot internally lately about how we interview, and thought folks might be interested in what our interview process looks like for engineering roles, and why.
Rethinking the Technical Interview
Much has already been said about interviewing practices in the software industry. We’ve seen or heard it all: from the opaque-by-design to the downright strange (I was once abandoned in a conference room, alone with only a bag lunch and no idea what was happening next).
No one likes these types of interviews. The people conducting them don’t, and neither do the candidates. After all, interviewing for a job is stressful enough without a process designed to pressure, confuse, or intimidate.
Our only goals when interviewing candidates are:
- Identify the best candidates and invite them to join our company.
- Provide all candidates with a positive experience.
Since I joined CircleCI, we’ve been working to design an interview process that we’d like to go through ourselves, one that shows respect to candidates while helping us find the best additions to our team.
Interview Structure
We designed a 4-step technical interview that follows the initial screen all candidates go through. All our interviews follow the same sequence, with each stage designed to accomplish a specific assessment:
1. Phone screen. The hiring manager for the position will talk with the candidate to assess experience, communication abilities, and experiences dealing with their own code in production.
2. Micro-skills take-home problem. The candidate completes a coding problem and reviews it with the interviewer. Talking through the problem with the candidate helps us assess ability as well as the candidate’s thought process, and how their priorities and values in writing software line up with ours. Many companies we know skip this stage for manager roles, since they won’t be writing much code. We include it because it’s so important to find out if the candidate’s values and priorities align with those of the team they will be managing.
3. Macro-skills design problem. This conversation with the senior members of the engineering team focuses on how the candidate can design and evolve a solution when faced with increasingly complex requirements. There’s not a right or wrong answer, but rather it serves as a sandbox in which we get to see where the boundaries of their abilities are. Our scoring of this test will depend on the level of the position they are interviewing for.
4. On-site interview. The goal of the on-site interview is for both parties to achieve certainty that the candidate should be a part of the CircleCI team. Candidates pair with members of the engineering team, and work together on problems/projects in the actual CircleCI codebase. This gives both our team and the candidate a taste of what it will be like to work together, and provides us with a sense of how this person approaches problems, collaborates, and communicates over the course of a more complex interaction. By the end of this pairing, both parties should feel able make a confident decision about fit. Update, March 2019: We have now moved to a fully remote interview process, to reflect our actual day-to-day collaboration. We still use this interview stage, but conduct all interviews remotely.
Creating a Respectful Process
We wrote an internal hiring guide that covers the values we try to uphold when doing interviews, and I share some of it below. Showing respect for candidates is incredibly important to us, and it gets expressed in many ways. I consider these basic courtesies: having someone to greet the candidate when they arrive, or not leaving them alone, wondering what is going to happen next. A few of our interviewing guidelines are:
Be kind. Interviewing is highly stressful, and most companies put candidates through ridiculous hoops that only make it worse. Let’s be an example of how things can be better.
Test for real skills. We are specifically against “whiteboard interviews”. We hold ourselves to never, under any circumstances, putting a candidate into a situation that does not represent the day-to-day reality of the position for which they are interviewing. Discussing design options on a whiteboard is fine, expecting candidates to regurgitate complex algorithms from memory is not.
Respect candidates’ time. We start and end interviews on time, and aim to give candidates feedback within 24-48 hours.
Do your homework. Do not make the candidate walk through their work history. Instead, review the candidate’s resume prior to your interview and be prepared to discuss interesting or relevant experience.
Spend most of your time listening. If you are talking more than 20% of the time, you are not learning as much as you can from the candidate.
Use Words Wisely
This starts with the job description. We do our best to avoid “bro culture” language or other highly-biased language. In all our writing, including job descriptions, we strive for inclusive language that emphasizes collaboration, trust, and learning.
If you want to see if your job descriptions are sending a message you don’t intend, Joblint and Gender Decoder are a couple of great tools for parsing biased language.
For example, this is our ad for Senior Backend Engineer run through Gender Decoder:
We purposely use gender-neutral and feminine language in our engineering job descriptions to create a more inclusive recruiting process. Studies have shown that that women are less likely to respond to an ad that has overly masculine or aggressive language (despite being qualified), whereas men will apply regardless.
Continuous Learning
This process continues to evolve and be informed by our experiences. For example, right now with an average .5% conversion rate for applicants, we’d need to look at up to 4,000 candidates to hire another 20 engineers.
Our ultimate goal is to have the best people here, doing their best work. As we continue to learn and refine, we’d like to improve our conversion rate while maintaining our high standards for competence and the quality of candidates’ experience.
Feedback on this process from our interviewees has generally been positive, and we are always open to feedback of all kinds as we refine our interview process. If you’ve been through our process and have feedback, I’d love to hear from you. Email me at jeff@circleci.com.
And if you are interested in applying for a job here, we’d love to speak to you too! Take a look at our open roles here.
Interested in learning more about what it’s like to work here? Read:
