The Problem

Only recent JVM versions allow a clean and manageable way of setting JVM limits in a containerized world. Using -Xmx settings involves rebuilding the images.

This is how JVM based containers look like in a K8s cluster:

The Requirements

  1. JVM Containers should have a small memory footprint
  2. JVM Memory Configuration has to be done via K8s resource limits

The Solution

The Docker Image

Dockerfile:

# make use of JDK 8u131 at minimum
FROM openjdk:8u131-jre-slim-stretch

...

ENV JAVA_OPTIONS="-server $JAVA_OPTIONS"

# engage the magical stuff available since JDK 8u131  
ENV JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions $JAVA_OPTIONS"

# calculate heap size from limits of cgroups instead of the nodes lmits
ENV JAVA_OPTIONS="-XX:+UseCGroupMemoryLimitForHeap $JAVA_OPTIONS"

# only make use of half of the available memory for the heap - GC will need this
ENV JAVA_OPTIONS="-XX:MaxRAMFraction=2 $JAVA_OPTIONS"

# print out the memory settings on booting up the JVM
ENV JAVA_OPTIONS="-XshowSettings:vm $JAVA_OPTIONS"

# make use of G1
ENV JAVA_OPTIONS="-XX:+UseG1GC $JAVA_OPTIONS"

ENTRYPOINT exec java $JAVA_OPTIONS -jar /my-jar-file.jar

To get the JVM in a container friendly mood we have to enable some special flags introduced in version 8u131.

Afterwards the JVM will base its heap memory limit calculations on cgroup limits (and not on node limits).

Do not try to calculate the limits on your own and set them via -Xmx. The goal is to keep those boundaries flexible. K8s should be the one managing those sizes.

K8s

my-app.yaml:

apiVersion: apps/v1
kind: Deployment
...
spec:
  ...
  template:
    ...
    spec:
      containers:
        - name: my-app
          image: my-registry/my-image:1.0.0
          resources:
            limits:
              memory: "1.0Gi"
...

You can set the limits in your k8s file. You can even overrule $JAVA_OPTIONS like this:

...
containers:
- name: my-app
  image: my-registry/my-image:1.0.0
  env:
    - name: JAVA_OPTIONS
      value: "-XX:MaxRAMFraction=4"
... 

So your K8s yaml has become your place to go to tweak your JVM. You do not need to recreate your Docker Image for doing this. That’s awesome.

One final note to the flag -XX:MaxRAMFraction. The factor 2 means the Heap gets about the half of the available memory. This is not suitable for all situations. The default value is 4. Working with the factor 1 does not work out. K8s will kill PODs due to its memory limits when GC kicks in. As it looks GC is in desperate need of OS Caches.