Avoiding and Debugging Java Memory Errors
How to avoid and debug Java memory errors on CircleCI.
The Java Virtual Machine (JVM) provides a portable execution environment for Java-based applications. Without any memory limits, the JVM pre-allocates a fraction of the total memory available in the system. CircleCI runs container based builds on large machines with lots of memory. Each container has a smaller memory limit than the total amount available on the machine. This can lead to the JVM seeing a large amount of memory being available to it, and trying to use more than is allocated to the container.
This pre-allocation can produce Out of Memory (OOM) errors, which are difficult to debug because the error messages lack detail.
You can see how much memory your container is allowed to use by reading the file
Recent versions of Java (JDK 8u191, and JDK 10 and up) include
UseContainerSupport which defaults on. This flag enables
the JVM to use the CGroup memory constraints available to the container,
rather than the much larger amount of memory on the machine.
Under Docker and other container runtimes, this will let the JVM more accurately
detect memory constraints, and set a default memory usage within those constraints.
You can use the
MaxRAMPercentage flag to customise the fraction of available RAM that is used,
In CircleCI, containers are run using Nomad. Nomad does set CGroup memory limits, but doesn’t provide enough CGroup memory information to the container for the JVM to detect the container memory constraints. This means the JVM will set it’s memory as a fraction of the total amount of RAM on the system. Nomad currently has an enhancement request open to provide this information. Once that is added, container builds in CircleCI will automatically pick up their container memory limits.
Manual memory limits
Using Java Environment Variables to Set Memory Limits
You can set several Java environment variables to manage JVM memory usage. These variables have similar names and interact with each other in complicated ways.
The table below shows these environment variables, along with the precedence levels they take when using different build tools. The lower the number, the higher the precedence level, with 0 being the highest.
|Java Environment Variable||Java||Gradle||Maven||Kotlin||Lein|
The above environment variables are listed below, along with details on why to choose one over another.
This environment variable takes precedence over all others. It is read directly by the JVM and overwrites all other Java environment variables, including command-line arguments. Because of this power, consider using a more specific Java environment variable.
_JAVA_OPTIONS is exclusive to Oracle.
If you are using a different runtime,
ensure that you check the name of this variable.
if you are using the IBM Java runtime,
then you would use
This environment variable is a safe choice
for setting Java memory limits.
JAVA_TOOL_OPTIONS can be read by all Java virtual machines,
and you can easily override it
with command-line arguments
or more specific environment variables.
This environment variable is not read by the JVM. Instead, several Java-based tools and languages use it to pass memory limits to the JVM.
This environment variable is exclusive to Clojure.
to pass memory limits to the JVM.
JVM_OPTS does not affect the memory of
nor can it directly pass memory limits to Java.
lein’s available memory,
To directly pass memory limits to Java,
This environment variable is exclusive to
This environment variable is exclusive to Gradle projects.
to overwrite memory limits set in
This environment variable is exclusive to Apache Maven projects.
to overwrite memory limits set in
Debugging Java OOM Errors
Unfortunately, debugging Java OOM errors often comes down to finding an
code 137 in your error output.
Ensure that your
-Xmxn maximum size is large enough for your applications to
completely build, while small enough that other processes can share the
remaining memory of your CircleCI build container.
If you are still consistently hitting memory limits, consider increasing your project’s RAM.