Tuning the JVM

Martini is a web application configured to run using Apache Tomcat, a Java Servlet Container. Tomcat in turn, runs on top of a Java Virtual Mahchine (JVM). This guide will discuss the various JVM configuration options that can help, directly or indirectly, to improve Martini's performance.

Generic JVM configuration?

While it may be worthwhile to have a one-size-fits-all JVM configuration, it's recommended to set up your JVM depending on how your organization uses Martini. This page includes tips intended to assist you in deciding which parameters you should use and which values would be appropriate for them.

Tuning requires more than just a basic knowledge of Java. The person responsible for tuning the JVM should have a thorough understanding of the production environment, and JVM internal memory management. Expertise in these topics is important, as attempting to tune an application without the technical know-how may do more harm than good.

JVM configuration is a broad topic, so for the meantime, this article will focus on topics and configuration options related to the JVM garbage collector (GC), a tool that greatly contributes to JVM performance degradation.

JVM garbage collector and heap

Among the benefits of running an application on a JVM is that under the covers, it uses a memory management tool called a garbage collector. This tool frees up unused Java objects¹ in the heap. The heap is where all 'live' and 'dead' objects reside; it is a "runtime data area from which memory for all class instances and arrays is allocated"².

However, while the idea of a garbage collector sounds appealing, it comes at a cost; cleaning unused Java objects in the heap space causes all application threads to stop until the cleaning operation completes. This is called a stop-the-world (STW) event. The duration of the pause depends on the machine, JVM configuration, and the number of objects stored in memory. Simply put, the STW events of the garbage collector can greatly degrade the overall performance of an application by stalling the application and consuming what otherwise could have been used for application throughput.

The heap is divided into two important parts: (1) young generation and (2) old generation.

According to Oracle's document on garbage collection basics:

"The young generation is where all new objects are allocated and aged. When the young generation fills up, this causes a minor garbage collection. Minor collections can be optimized assuming a high object mortality rate. A young generation full of dead objects is collected very quickly. Some surviving objects are aged and eventually move to the old generation.

The old generation is used to store long surviving objects. Typically, a threshold is set for young generation object and when that age is met, the object gets moved to the old generation. Eventually the old generation needs to be collected. This event is called a major garbage collection.

Often a major collection is much slower because it involves all live objects. So for responsive applications, major garbage collections should be minimized. Also note, that the length of the stop-the-world event for a major garbage collection is affected by the kind of garbage collector that is used for the old generation space."

The two most influential factors that affect garbage collection performance are:

• Total available memory

  A common resolution for improving the performance of any application is by giving it more memory to work with. By giving the JVM more memory, it will take longer for the heap to fill up; thus, collections will occur less often. But a bigger heap can also mean the presence of more objects³; thereby making garbage collection slower. From this, we can say that "throughput is inversely proportional to the amount of memory available."⁴

• Proportion of the heap dedicated to the young generation

  "The bigger the young generation, the less often minor collections occur. However, for a bounded heap size, a larger young generation implies a smaller old generation, which will increase the frequency of major collections. The optimal choice depends on the lifetime distribution of the objects allocated by the application."⁴

Tuning the Java heap

One of the easiest ways to boost Martini's performance is by giving it more memory; however, the more memory you allocate, the more garbage the collector will have to clean. The JVM heap size determines how long and how often the garbage collector spends on cleaning unused Java objects. The goal in choosing the optimal heap size is to minimize the time spent on garbage collection versus the time spent on the application.

The JVM arguments for setting the heap size are:

• -Xms<size> which sets the initial and minimum heap size
• -Xmx<size> which sets the maximum heap size

For example:

java -Xms2g -Xmx2g Martini

Equal values for -Xms and -Xmx

Setting the -Xms and -Xmx size with equal values will yield higher performance throughput than setting the values differently as this prevents the JVM from spending time to increment the allocated heap.

Tuning the JVM garbage collector

When increasing the allocated JVM memory is no longer a valid resolution for improving the performance of your application, you can start tuning your garbage collector to improve the pause time or frequency of stop-the-world events. In general, to do this you must:

1. Choose the type of garbage collector that suits your production environment.
2. Profile the results and see whether your changes were satisfactory.

Rinse and repeat until you get the JVM garbage collector configuration that is right for you.

Choosing the right JVM garbage collector

According to the Java documentation:

"When does the choice of a garbage collector matter? For some applications, the answer is never. That is, the application can perform well in the presence of garbage collection with pauses of modest frequency and duration. However, this is not the case for a large class of applications, particularly those with large amounts of data (multiple gigabytes), many threads, and high transaction rates."

Available collectors

There are two types of collectors: a throughput-oriented collector, and a low pause collector. Throughput is defined by the application total time that is not spent in the garbage collector. Pause-time is defined by how long the stop-the-world event takes during garbage collection.

A simple way to choose between the different types of garbage collection is to consider the requirements of your production environment. If you want Martini to perform as much work as possible and can withstand recurring longer pauses then a suitable garbage collector is a throughput-oriented collector. If, however, your production environment prioritizes low latency then the low-pause collector may be more suitable.

Throughput-oriented collectors

• Serial collector

  This is used to perform all garbage collection work using a single thread. This works best on a single processor machine as it cannot take advantage of multiprocessor hardware, although it can be useful on multiprocessor machines if the dataset is less than 100 MB. This is enabled by the JVM argument -XX:+UseSerialGC.

• Parallel collector (or throughput collector)

  This performs minor collections in parallel, which significantly reduces garbage collection overhead. This garbage collector is best suited if your production environment runs on a multiprocessor machine with medium to large sized data sets. This is enabled by the JVM argument XX:+UseParallelGC. This argument, however, only uses parallel garbage collection on minor collections; in major collections you will need to explicitly enable it with -XX:-UseParallelOldGC.

Low-pause collectors (response time-oriented collectors)

• Concurrent mark-and-sweep collector

  This collector is designed to perform the cleaning work concurrent with the application. This still requires the application to pause for certain cleaning phases. This is enabled by the -XX:+UseConcMarkSweepGC JVM argument.

• Garbage first collector

  Another alternative to the concurrent garbage collector is the new G1. This is the default garbage collector of Java 9 and is said to replace the CMS GC. This is enabled by the -XX:+UseG1GC argument.

Profiling results

After choosing the garbage collector that suits your production environment, the most reliable way to know whether you made the right choice or not is by profiling Martini. Don't get fixated on your JVM configuration just yet. In the world of performance tuning, there is one iron-clad rule: "Never guess. Profile the application."

Profiling rules

When testing your configuration, it is important to follow these points to ensure the accuracy of results:

• Perform tests in a controlled environment. This means that you have to test Martini in a dedicated environment where nothing but Martini affects the overall performance.
• Perform tests that mimic real world scenarios.
• Learn how to monitor the garbage collector's logs.

Testing procedure

The procedure for profiling the configured application is generally a repetitive process and roughly entails the following steps:

1. Determine the production environment's requirements.
2. Determine the suitable garbage collector for the production environment.
3. Configure the garbage collector suitable for the production environment.

4. Enable garbage collector logging on the JVM. This can be as simple as using the JVM option:

   -XX:+PrintGCDetails-XX:+PrintGCDateStamps-XX:+PrintGCTimeStamps -Xloggc:gclog.log
   

5. Conduct tests using a test case that mimics a real world scenario. A good example is executing REST calls.

6. Analyze the garbage collection logs. You can make use of user-friendly automated GC log analyzers to do the job for you.
7. Decide whether the current configuration is satisfactory for your production environment. If application pause duration is too long or if garbage collection occurs too frequently, then you might have to tune garbage collection further. In this case, repeat all steps starting from step two.

Example

To save some time, the example below will use one of TORO's existing tuning examples – specifically about optimizing the throughput of web services. This will be using the configuration and testing guidelines described in the linked document.

To perform this test:

• Turn off both Tracker and Monitor;
• Modify the test script to return a 29.346 KB JSON response; and
• Measure performance using throughput.

The test case will show the difference between two JVM arguments: -Xms1g -Xmx1g and -Xms2g -xmx2g. TORO has already run the test, and the results are described in the chart and table below:

The difference between these two configurations is only minimal. This is mainly due to the fact that the only significant process that consumes memory during the test is the receiving of REST requests and serving of REST responses. In a real world scenario, Martini will have more work to do than just receiving and responding to REST requests.

Resources

Not enough of a performance boost?

Sometimes, the default configuration of the garbage collector is not enough or adding more memory is not an option. In this case, configuring the heap's young generation is an option; this is considered the second-most influential factor in the performance of the garbage collector.

To guide you in your journey to JVM optimization, learn more using the following links:

• Tuning Java Garbage Collection for Web Services
• Garbage Collection Optimization for High-Throughput and Low-Latency Java Applications
• Java Platform, Standard Edition HotSpot Virtual Machine Garbage Collection Tuning Guide

---

1. Java objects that are no longer referenced at any succeeding point of the program. ↩

2. Oracle Corporation. (n.d.). Chapter 2. The Structure of the Java Virtual Machine. Retrieved January 7, 2019, from https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-2.html ↩

3. Though this is not always the case. ↩

4. Oracle Corporation. (n.d.). Sizing the Generations. Retrieved March 13, 2019, from https://docs.oracle.com/javase/8/docs/technotes/guides/vm/gctuning/sizing.html#sthref22 ↩↩