Along with choosing a programming language, an editor, CI/CD, and a VCS, choosing a build tool is one of the most important and impactful decisions we can make as developers. You use it every single time you test or compile your app, or bundle it for deployment to production.
Given how important the build tool is, choosing it can be a daunting endeavour, something I will try to make easier with this article.
I will cover a few things:
- What build tools do and some examples of tools
- Why you might want to consider a build tool - either when starting a project from scratch, or when you are migrating an existing project
- How you should test any build tool to make sure it is the best one for you, your team, and your project
Note: This post assumes that you need a build tool for a commercial project (one that earns you money). A hobby or educational project will have different requirements that will affect your choice of tools. In that case, you do you, and do what you will.
Spoiler alert: I am biased and I believe the default, most widely adopted tool for your platform is most likely the best one for you, too.
What is a build tool?
A build tool ties together a number of utilities and scripts for compiling, packaging, testing, running, and releasing your application. Often they let you do dependency management. Some let you script more steps and plugins, like code coverage and security scanning. Your build tool serves as a single point of contact with your CI/CD system, replacing the individual tools that had to be run manually.
Examples of build tools
Some popular build tools include NPM and Yarn - for Node, Gradle, and Maven. Ant is a build tool for Java, Rake works with Ruby, and Make is a build tool for Unix and Linux systems. A particular tool may be tied to the platform and programming language they work with, like Gradle for JVM, and Rake for Ruby-based projects. This means that they bring useful abstractions for dealing with their respective platforms. Some are also designed to be language agnostic, like Make or Bazel, which allows them to be used by organizations developing on multiple platforms and languages.
Tools also differ in who makes and maintains them. Make and Rake are “proper” open source projects, backed only by the community. NPM and Gradle are backed by their respective for profit organizations, which charge for either enterprise features, support, or services. Tools like Bazel and Buck were developed by big tech companies (Google and Facebook, respectively) as internal tools that they decided to release as open source projects without making money from them.
Why pick a build tool?
Now that we have covered some examples of the build tool landscape, we can look at the possible reasons you may be picking a tool.
If you are starting a project from scratch
The first reason for selecting a build tool is starting a project from scratch. You need a build tool if you don not want to call all low level commands manually. The manual effort inevitably ends with a set of untested, half-baked bash scripts, which is a terrible idea.
I suggest that you go with the default tool, which is whatever your editor, or project generator, or template uses. For example, for Android projects the default is currently Gradle. Alternatively, you could go with a general purpose tool like Buck or Bazel, or a different JVM-specific tool like Maven.
If you are migrating an existing project
The second reason is that sometimes the default build tool does not work for you. This could be because you are facing performance issues: builds being slow or running out of RAM. Sometimes the builds are flaky and non-deterministic, causing you to spend a lot of time tweaking the build scripts and CI configuration in an attempt to fix the builds.
In that case, changing to a different tool, maybe one that has been hailed as extremely fast and embraced by large tech giants in blog posts and conference talks, can be compelling. It is also what I have done in the past.
Steps for picking a build tool
I recommend you take the build tool through a series of tests, to see how it works for your project. These tests go from less to more involved, with latter stages potentially taking days and even weeks of your work. Again, picking a build tool is a big decision, so tread carefully. Take some time to document the entire process and outcomes of each of the tests.
Choosing a build tool should not be a lone endeavour, so make sure you involve your team members early and often. You will need their buy-in for any new tool or process to be successful. Build tools are not an exception.
Start with a problem statement
Consider what you are looking for in a build tool. There are many valid reasons to select a tool, but ‘we should adopt it because BigCo Inc. uses it’ is not one of them. This test could take a few hours to complete, often less.
Here are some ideas for good problem statements:
-
Developer productivity works both for existing team members, as well as onboarding new hires, and for productivity across your internal teams and departments.
-
Performance of builds is especially relevant when migrating your existing project to a new build tool because of degraded performance of builds. Sometimes that is a legitimate concern, but other times this can be remedied by tweaking and optimizing the usage of your existing tools. For example, optimizing your CI/CD build configuration can also increase performance.
-
Unification across the org means establishing a common build infrastructure team and staffing it with experts who maintain and tweak the build tools and infrastructure across the entire org. For this strategy to work, your organization should be large enough to sustain an infrastructure team. In that case, offloading some of the effort to this team makes sense. See also: How to be a CI/CD Engineer.
If you strive for developer productivity and the other two issues are not a problem, your team may be best served by using the default option. While every Android developer will know their way around Gradle, fewer will be adept in Buck or Make.
Evaluate the documentation
Once you know the primary motivation for your choice of build tool, you can start with the second consideration - evaluating the docs and other learning materials. This should take longer than defining your problem statement, but you should not have to spend more than a day per build tool.
What you are looking for in docs can vary, but you are likely to end up asking some of these questions:
-
What were the motivations and design considerations taken when creating the tool? This might be part of docs or a separate blog post, but it is great to get into the author’s head to see why they wanted to create the tool in the first place. Maybe it is especially optimised for building microservices in a monorepo, and that is exactly what you need.
-
Are the docs complete? Check various pages of the docs to see if you find any important documents that are still stubs, like the How to Test Your App section. It depends on your tolerance for risk, but incomplete docs can signal a tool is not ready for prime time.
-
What is the support like across various channels? Documentation is not just
docs/. Is the community large, active, helpful, and supportive? Is there a code of conduct and moderators to uphold it? Check for open and closed issues on GitHub, the popularity of the tool on StackOverflow, and any other outlets you might use, like Slack teams or Discord servers. If the docs are lacking, a helpful community is certainly necessary. -
How does the code look? Check for any missing comments in important files. Is the tool integrated with a CI/CD service? Is the main branch passing? When was the latest release? Are there a lot of examples for different use cases?
If the tool meets or exceeds your expectations for all these considerations, continue. Else, go with the default.
Evaluate a sample project
Now it is time to check out some of their samples or tutorials to see how they feel. Clone a sample project (or even better, follow a step-by-step tutorial), and explore it. This can easily take a day for a single tool if you want to do it thoroughly.
Get the project building locally and then run tests, change steps, add dependencies, write additional scripts, and integrate with the third-party services your team uses. Add whatever you think your real project will need.
How does the tool behave? Is everything as you feel it should be? Is the sample project even there for your platform and language choice, or did you have to create it from scratch?
At this point you should also get the sample project set up with your CI/CD tool, and make sure it builds and runs the tests nicely. Your default build tool might be built and supported out of the box, but with less popular tools you might need to install it on your build machine. Time to install and difficulty will vary. Check how long installation and setup take. Is the process smooth, or did you experience any glitches?
Once you have the minimal sample project building on your CI platform, you can also go back and re-implement the same sample application across the multiple build tools that you are evaluating. If you are looking for speed and performance improvements, your CI platform should give you an easy way to test that, and compare the results.
Were the preceding steps straightforward to do? Did you struggle significantly trying to get it building or testing? Did you run into documentation problems? Any of these issues might be just the tip of the iceberg compared to the next consideration.
Migrate to real project (or as close to a real project as possible)
If you are still happy and still trying to decide, it is time to take a leap and use this tool for real. If you are migrating an existing project, you know what you need to build. If you are not, and researching for a project built from scratch, expand on the sample to include everything you might need.
The checks you will perform here will be similar to the previous step with the sample project, but with the benefit of evaluating your tool using a real-world scenario. The most important thing to note is timeboxing this exercise. Take a few days, a week - however long you might need. Consult the team, and add some padding like you would in sprint planning.
By the end of this exercise, you should be ready to make a decision, and convince the team that the build tool you have selected is the right choice.
Get agreement from your team
Team buy-in for developer tooling is everything; with something as impactful as a build tool, it is even more so. Talk to your team and present them with the results of your tests. Make sure they know the differences that set the tool apart, and the intricacies and specifics that make it the right choice for your team.
How long did it take for to learn how to use the tool? Take that time and multiply it by the number of people on your team. Is it enough to convince your manager and stakeholders? If you are confident in your choice and have the numbers to prove it, you should be in a good spot. Go forth and build!
Conclusion
A developer’s build tool is as important a choice as a programming language and a framework. Knowing and understanding why you picked something will have a great impact on you and your team’s productivity. I have provided these considerations in an intentionally cautious way. They favor the status quo and the default option for your platform. In my experience, great community support, documentation, and samples projects are what you really need for success as a developer. A great build tool will help you, your team, and your product succeed too.
