GraalVM Native Image: Real Life

Photo by Markus Spiske on Unsplash

In ANNA, we have several external integrations with partners, for example, to send emails. Test environments usually have some limitations, can be out of work, and process requests slowly. It leads to additional issues with end-to-end test stability, especially if you have complex scenarios. To eliminate them, we use mockserver to mock external requests and do not bet our lives to the durability of the partner’s test stands.

Mockserver has all the necessary functionality, and we prefer to use a third-party solution, not write ourselves. As the number of tests increased, we began to notice that the mockserver response time increased. Also, it requires a lot of memory and becomes CPU-intensive. In peak, it consumes more than 4 four CPU and 4 GB memory.

First, we tried to optimize some things that relative to mockserver. Turn-off in-memory logs, clear expectations after each test, tune JVM to achieve more predicted latency (we expect less than 1s for any call, in ideal less). These things help us to make mocks more stable, but they still consume much more resources that we want to cut.

That moment we decided to try GraalVM Native Image to decrease memory footprint at least.

GraalVM Native Image allows you to ahead-of-time compile Java code to a standalone executable, called a native image. It uses SubstractVM under the hood and the resulting program has faster startup time and lower runtime memory overhead compared to a Java VM.

Think that is what we need precisely. Let’s try to make mockserver native image.

The first attempt

Easy peasy as I thought. Create Dockerfile to grab mockserver and repack it with GraalVM to Native Image:

#original image
FROM jamesdbloom/mockserver:mockserver-5.10.0 as build
#repack with graalbm
FROM oracle/graalvm-ce:20.1.0-java11-ol8 as native-build
# copy in jar
COPY --from=build /opt/mockserver/mockserver-netty-jar-with-dependencies.jar /
WORKDIR /opt/graalvm
#install native-image
RUN gu install native-image
RUN native-image --verbose --no-server -Dnative-image.xmx=6g \
    --static --allow-incomplete-classpath --no-fallback \
    --report-unsupported-elements-at-runtime \
    -jar /mockserver-netty-jar-with-dependencies.jar \
    -Dfile.encoding=UTF-8 -Dio.netty.noUnsafe=false \
    mockserver-native
#runtime image
FROM alpine
EXPOSE 8000
WORKDIR /opt/mockserver
COPY --from=native-build /opt/graalvm/mockserver-native .
CMD ["/opt/mockserver/mockserver-native", "-logLevel", "INFO", "-serverPort", "8000"]

What I did first according to guides on the internet:
- gave around 6Gb to compilation process (in fact only three was used) and it could be necessary to increase docker daemon memory limits
- added some useful options to build a self-executable image with incomplete classpath and prevent building fallback image that requires JVM.

The first result was not successful, but I expect it:

Error: Classes that should be initialized at run time got initialized during image building:
 org.slf4j.LoggerFactory was unintentionally initialized at build time.

So, ok. Some classes we should initialize at build time and I added explicit instruction to build them accordingly:

RUN native-image ... --initialize-at-build-time=org.slf4j ...

That worked, and the image was built, and I tried to launch it but got an error that class org.mockserver.logging.StandardOutConsoleHandler was not found.

The second attempt

Reflection is hard, and the AOT compiler does not know which classes we should include in native image and which not. It is fair, and there is a possibility to create a JSON file with a reflection definition and use it via the build process. For example, this file can fix my problem:

[
  {
    "name" : "org.mockserver.logging.StandardOutConsoleHandler",
    "allDeclaredConstructors" : true,
    "allPublicConstructors" : true,
    "allDeclaredMethods" : true,
    "allPublicMethods" : true,
    "allDeclaredClasses" : true,
    "allPublicClasses" : true
  }
]

But fill it manually is a messy and monkey job, and there is another way to create it. Meet native image java agent that is included. Some tutorials that you could meet on the internet still provide this line to connect the agent to your app:

-agentlib:native-image-agent=trace-output=/path/to/trace-file.json

The other way is using another agent option:

-agentlib:native-image-agent=config-output-dir=/path/to/config-dir/

To fill the reflection file accordingly (and not only it but also resources and JNI), you should run your app with much as possible different cases. I started with a launch app only with some init config:

RUN timeout --signal=SIGTERM 10 \ 
    java -agentlib:native-image-agent=trace-output=trace-log.json \
    -Dio.netty.noUnsafe=false -Dfile.encoding=UTF-8 \
    -jar /mockserver-netty-jar-with-dependencies.jar \
    -logLevel INFO -serverPort 1080 \ 
    || true

Trace log requires to be processed by a particular configuration tool. I tried recommended way like:

native-image --tool:native-image-configure
native-image-configure process-trace \
  --output-dir=/path/to/config-dir/ /path/to/trace-file.json

but there is no tool as native-image-configure . I used the config-output-dir option and forgot about this little wtf until the next issue was faced.

The generated reflection file did not include the necessary class org.mockserver.logging.StandardOutConsoleHandler. And I needed to add it manually or find a way to use native-image-configure .

It is strange, but the configuration tool is included as code into the native-image package but not as a tool. To use it we should run it as a java program:

RUN java --add-exports jdk.internal.vm.compiler/org.graalvm.compiler.phases.common=ALL UNNAMED --add-exports jdk.internal.vm.ci/jdk.vm.ci.meta=ALL-UNNAMED -cp /opt/graalvm-ce-java11-20.1.0/lib/graalvm/svm-agent.jar:/opt/graalvm-ce-java11-20.1.0/lib/svm/builder/svm.jar com/oracle/svm/configure/ConfigurationTool \
    generate --reflect-input=reflect-custom.json \
             --trace-input=trace-log.json \
             --output-dir=./mockserver/native-configure

And voila, we have a reconfigured reflection file.

I relaunched mocks, and they did not run again.

The third attempt

The issue was mockserver uses a javascript engine to check pattern properties (it does not matter why they do it via this way) and evaluates regular expressions via JS scripts. It requires to startup JS Engine and does some dynamic code evaluation, and SubstractVM does not support it. I saw an error like this:

Unsupported method java.lang.ClassLoader.defineClass1(ClassLoader, String, byte[], int, int, ProtectionDomain, String) is reachable

We were lucky because the JS engine is required only for checking regular expressions, but I should find a way to no use pattern properties.

After some code reading, I found that only two places have a model specification that requires patterns:

{
   "type": "object",
   "additionalProperties": false,
   "patternProperties": {
      "^\\S+$": {
         "$ref": "#/definitions/stringOrJsonSchema"
      }
   }
}

I replaced it to more straightforward schema:

{
   "type": "object",
   "additionalProperties": true
}

with following Docker command that repacks jar resources:

RUN jar xvf mockserver-netty-jar-with-dependencies.jar
COPY keyToValue.json keyToMultiValue.json org/mockserver/model/schema/
RUN rm ./mockserver-netty-jar-with-dependencies.jar && \
    jar cmf META-INF/MANIFEST.MF ./mockserver-netty-jar-with-dependencies.jar *

And after the next launch, it worked!

The final attempt

I deployed new mocks to our test environment and faced that some expectations suddenly disappear in the middle of testing. If I run mocks with JVM, it did not happen.

I could not find a real reason for that problem, but I think that it was because many classes were inited differently for the native image.

The solution was simple, but not pretty. I ran mockserver on the test environment with a native image java agent. Then ran tests and collected trace logs from real execution. After that, I used this trace log while building a Docker image. And it works!

The final command to build a native image:

RUN native-image --verbose --no-server -Dnative-image.xmx=6g \
    --static --allow-incomplete-classpath --no-fallback \
    --report-unsupported-elements-at-runtime \
    --initialize-at-build-time=org.slf4j \
    -jar /mockserver-netty-jar-with-dependencies.jar \
    -Dfile.encoding=UTF-8 -Dio.netty.noUnsafe=false \
    -H:IncludeResourceBundles=
com.sun.org.apache.xml.internal.res.XMLErrorResources,com.sun.org.apache.xerces.internal.impl.msg.XMLMessages \
    -H:ReflectionConfigurationFiles=native-configure/reflect-config.json \
    -H:ResourceConfigurationFiles=native-configure/resource-config.json \
    -H:JNIConfigurationFiles=native-configure/jni-config.json \
    -H:ConfigurationFileDirectories=native-configure \
    mockserver-native

As you can see, I included some additional resources and passed all configuration files explicitly.

The final notes

Community edition has other limitations:
- serial garbage collector only
- a limited number of GC options
- no profile optimizations

Just keep it in mind.

The results

1. Memory decreased from 4GB to 1GB (with additional optimizations for mock server)
2. CPU dropped from 4 vCPU to 0.5 vCPU max (with other optimizations for mock server)
3. 99 percentile decreased from 10 seconds to less than 1 second

Native image is a really cool thing but you should be ready to pain in the ass while cooking it.