Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Java 9 compatibility #543

Closed
edeandrea opened this issue Jan 23, 2018 · 4 comments
Closed

Java 9 compatibility #543

edeandrea opened this issue Jan 23, 2018 · 4 comments
Assignees
@nebhale
Labels
bug jre
Milestone
v4.9

Comments

@edeandrea
Copy link

edeandrea commented Jan 23, 2018 •
edited

Hi - as specified in #495 many frameworks don't seem ready for Java 9 yet. I have a very small "Hello World" spring boot app using Spring Boot 2 with Java 9. This app runs fine locally, yet when I push to cloud foundry using build pack 4.7 or higher (I've tried with 4.7, 4.7.1, & 4.8) the application blows up at startup.

When I run locally (compiled & running on Java 9) I see:

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::             (v2.0.0.M7)

2018-01-23 09:36:56.196  INFO 7511 --- [           main] c.a.c.c.ConnectTimeoutTestApplication    : Starting ConnectTimeoutTestApplication vLOCAL on MACHINE with PID 7511 (/path/to/my-app.war started by user in /path/to/)
2018-01-23 09:36:56.201 DEBUG 7511 --- [           main] c.a.c.c.ConnectTimeoutTestApplication    : Running with Spring Boot v2.0.0.M7, Spring v5.0.2.RELEASE
2018-01-23 09:36:56.203  INFO 7511 --- [           main] c.a.c.c.ConnectTimeoutTestApplication    : No active profile set, falling back to default profiles: default
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.cglib.core.ReflectUtils$1 (jar:file:/path/to/my-app.war!/WEB-INF/lib/spring-core-5.0.2.RELEASE.jar!/) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of org.springframework.cglib.core.ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
2018-01-23 09:36:58.673  INFO 7511 --- [           main] c.a.c.c.ConnectTimeoutTestApplication    : Started ConnectTimeoutTestApplication in 3.05 seconds (JVM running for 3.57)

The Java 9 warnings are just that - warnings. The Java 9 default is to permit the reflection to happen.

This is what my manifest looks like:

---
applications:
- name: my-app
  memory: 1024M
  path: target/archives/my-app.war
  instances: 1
  buildpack: https://github.com/cloudfoundry/java-buildpack.git#v4.8
  env:
    JBP_CONFIG_OPEN_JDK_JRE: '{ jre: { version: 9.0.+ } }'

Is there something else I need to do?

Here is the log output during the push:

   2018-01-23T08:21:43.93-0500 [STG/0] OUT Creating container
   2018-01-23T08:21:44.26-0500 [STG/0] OUT Downloading app package...
   2018-01-23T08:21:44.26-0500 [STG/0] OUT Successfully created container
   2018-01-23T08:21:46.70-0500 [STG/0] OUT Downloaded app package (13.6M)
   2018-01-23T08:21:46.70-0500 [STG/0] OUT Downloading build artifacts cache...
   2018-01-23T08:22:04.81-0500 [STG/0] OUT Downloaded build artifacts cache (118.2M)
   2018-01-23T08:22:04.81-0500 [STG/0] OUT Staging...
   2018-01-23T08:22:09.95-0500 [STG/0] OUT -----> Java Buildpack v4.8 | https://github.com/cloudfoundry/java-buildpack.git#7f50d3b
   2018-01-23T08:22:10.64-0500 [STG/0] OUT -----> Downloading Jvmkill Agent 1.12.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/jvmkill-1.12.0_RELEASE.so (found in cache)
   2018-01-23T08:22:11.02-0500 [STG/0] OUT -----> Downloading Open Jdk JRE 9.0.4_11 from https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-9.0.4_11.tar.gz (found in cache)
   2018-01-23T08:22:12.12-0500 [STG/0] OUT        Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.0s)
   2018-01-23T08:22:12.37-0500 [STG/0] OUT -----> Downloading Open JDK Like Memory Calculator 3.10.0_RELEASE from https://java-buildpack.cloudfoundry.org/memory-calculator/trusty/x86_64/memory-calculator-3.10.0_RELEASE.tar.gz (found in cache)
   2018-01-23T08:22:12.71-0500 [STG/0] OUT        Loaded Classes: 3514, Threads: 250
   2018-01-23T08:22:12.97-0500 [STG/0] OUT -----> Downloading Client Certificate Mapper 1.5.0_RELEASE from https://java-buildpack.cloudfoundry.org/client-certificate-mapper/client-certificate-mapper-1.5.0_RELEASE.jar (0.2s)
   2018-01-23T08:22:13.23-0500 [STG/0] OUT -----> Downloading Container Customizer 2.6.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-customizer/container-customizer-2.6.0_RELEASE.jar (0.2s)
   2018-01-23T08:22:13.46-0500 [STG/0] OUT -----> Downloading Container Security Provider 1.11.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-security-provider/container-security-provider-1.11.0_RELEASE.jar (found in cache)
   2018-01-23T08:22:13.69-0500 [STG/0] OUT -----> Downloading Spring Auto Reconfiguration 2.3.0_RELEASE from https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-2.3.0_RELEASE.jar (found in cache)
   2018-01-23T08:22:29.79-0500 [STG/0] OUT Staging complete
   2018-01-23T08:22:29.79-0500 [STG/0] OUT Uploading droplet, build artifacts cache...
   2018-01-23T08:22:29.79-0500 [STG/0] OUT Uploading droplet...
   2018-01-23T08:22:29.79-0500 [STG/0] OUT Exit status 0
   2018-01-23T08:22:29.79-0500 [STG/0] OUT Uploading build artifacts cache...
   2018-01-23T08:22:32.31-0500 [STG/0] OUT Uploaded build artifacts cache (118.2M)
   2018-01-23T08:22:42.96-0500 [STG/0] OUT Uploaded droplet (53M)
   2018-01-23T08:22:42.97-0500 [STG/0] OUT Uploading complete
   2018-01-23T08:22:43.00-0500 [STG/0] OUT Destroying container
   2018-01-23T08:22:43.43-0500 [CELL/0] OUT Creating container
   2018-01-23T08:22:43.76-0500 [CELL/0] OUT Successfully created container
   2018-01-23T08:22:43.90-0500 [STG/0] OUT Successfully destroyed container
   2018-01-23T08:22:54.70-0500 [CELL/0] OUT Starting health monitoring of container
   2018-01-23T08:22:54.78-0500 [APP/PROC/WEB/0] OUT JVM Memory Configuration: -Xss1M -Xmx498339K -XX:MaxMetaspaceSize=38236K -XX:ReservedCodeCacheSize=240M -XX:MaxDirectMemorySize=10M
   2018-01-23T08:22:57.48-0500 [APP/PROC/WEB/0] OUT   .   ____          _            __ _ _
   2018-01-23T08:22:57.48-0500 [APP/PROC/WEB/0] OUT  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
   2018-01-23T08:22:57.48-0500 [APP/PROC/WEB/0] OUT ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
   2018-01-23T08:22:57.49-0500 [APP/PROC/WEB/0] OUT  :: Spring Boot ::             (v2.0.0.M7)
   2018-01-23T08:22:57.48-0500 [APP/PROC/WEB/0] OUT  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
   2018-01-23T08:22:57.48-0500 [APP/PROC/WEB/0] OUT   '  |____| .__|_| |_|_| |_\__, | / / / /
   2018-01-23T08:22:57.48-0500 [APP/PROC/WEB/0] OUT  =========|_|==============|___/=/_/_/_/
   2018-01-23T08:22:58.02-0500 [APP/PROC/WEB/0] OUT 2018-01-23 13:22:58.017  INFO 20 --- [           main] c.a.c.c.ConnectTimeoutTestApplication    : Starting ConnectTimeoutTestApplication on 69fa52e2-a9f7-4102-4d97-e587983d77b0 with PID 20 (/home/vcap/app/WEB-INF/classes started by vcap in /home/vcap/app)
   2018-01-23T08:22:58.02-0500 [APP/PROC/WEB/0] OUT 2018-01-23 13:22:58.028  INFO 20 --- [           main] c.a.c.c.ConnectTimeoutTestApplication    : The following profiles are active: cloud
   2018-01-23T08:22:58.02-0500 [APP/PROC/WEB/0] OUT 2018-01-23 13:22:58.025 DEBUG 20 --- [           main] c.a.c.c.ConnectTimeoutTestApplication    : Running with Spring Boot v2.0.0.M7, Spring v5.0.2.RELEASE
   2018-01-23T08:22:59.56-0500 [APP/PROC/WEB/0] ERR WARNING: Please consider reporting this to the maintainers of org.springframework.cglib.core.ReflectUtils$1
   2018-01-23T08:22:59.56-0500 [APP/PROC/WEB/0] ERR WARNING: All illegal access operations will be denied in a future release
   2018-01-23T08:22:59.56-0500 [APP/PROC/WEB/0] ERR WARNING: An illegal reflective access operation has occurred
   2018-01-23T08:22:59.56-0500 [APP/PROC/WEB/0] ERR WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
   2018-01-23T08:22:59.56-0500 [APP/PROC/WEB/0] ERR WARNING: Illegal reflective access by org.springframework.cglib.core.ReflectUtils$1 (jar:file:/home/vcap/app/WEB-INF/lib/spring-core-5.0.2.RELEASE.jar!/) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
   2018-01-23T08:23:03.10-0500 [APP/PROC/WEB/0] ERR ResourceExhausted! (1/0)
   2018-01-23T08:23:03.10-0500 [APP/PROC/WEB/0] ERR Resource exhaustion event.
   2018-01-23T08:23:03.43-0500 [APP/PROC/WEB/0] OUT | Instance Count | Total Bytes | Class Name                                                                                                       |
   2018-01-23T08:23:03.43-0500 [APP/PROC/WEB/0] OUT | 40067          | 4122640     | [B                                                                                                               |
   2018-01-23T08:23:03.44-0500 [APP/PROC/WEB/0] OUT | 38723          | 929352      | Ljava/lang/String;                                                                                               |
   2018-01-23T08:23:03.44-0500 [APP/PROC/WEB/0] OUT | 673            | 331864      | [I                                                                                                               |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 14410          | 230560      | Ljava/lang/Object;                                                                                               |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 2396           | 217304      | [Ljava/util/HashMap$Node;                                                                                        |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 3737           | 149480      | Ljava/lang/ref/SoftReference;                                                                                    |
   2018-01-23T08:23:03.44-0500 [APP/PROC/WEB/0] OUT | 1062           | 2640424     | [C                                                                                                               |
   2018-01-23T08:23:03.44-0500 [APP/PROC/WEB/0] OUT | 7156           | 854704      | Ljava/lang/Class;                                                                                                |
   2018-01-23T08:23:03.44-0500 [APP/PROC/WEB/0] OUT | 20860          | 667520      | Ljava/util/concurrent/ConcurrentHashMap$Node;                                                                    |
   2018-01-23T08:23:03.44-0500 [APP/PROC/WEB/0] OUT | 9858           | 315456      | Ljava/util/HashMap$Node;                                                                                         |
   2018-01-23T08:23:03.44-0500 [APP/PROC/WEB/0] OUT | 2938           | 258544      | Ljava/lang/reflect/Method;                                                                                       |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 6087           | 243480      | Ljava/util/LinkedHashMap$Entry;                                                                                  |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 672            | 75264       | Lorg/springframework/boot/loader/jar/JarEntry;                                                                   |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 3587           | 114784      | Ljava/lang/ref/WeakReference;                                                                                    |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 1777           | 99512       | Ljava/util/LinkedHashMap;                                                                                        |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 2153           | 91832       | [Ljava/lang/String;                                                                                              |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 4233           | 237648      | [Ljava/lang/Object;                                                                                              |
   2018-01-23T08:23:03.45-0500 [APP/PROC/WEB/0] OUT | 222            | 208800      | [Ljava/util/concurrent/ConcurrentHashMap$Node;                                                                   |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 889            | 49784       | Ljava/beans/MethodDescriptor;                                                                                    |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 1759           | 70360       | Ljava/util/TreeMap$Entry;                                                                                        |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 2628           | 64784       | [Ljava/lang/Class;                                                                                               |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 923            | 51688       | Ljava/lang/invoke/MemberName;                                                                                    |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 1040           | 41600       | Lorg/springframework/util/ConcurrentReferenceHashMap$Segment;                                                    |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 1234           | 39488       | Ljava/lang/ref/ReferenceQueue;                                                                                   |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 2037           | 48888       | Ljava/beans/MethodRef;                                                                                           |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 1013           | 48624       | Ljava/util/HashMap;                                                                                              |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 1006           | 48288       | Lsun/util/locale/LocaleObjectCache$CacheEntry;                                                                   |
   2018-01-23T08:23:03.46-0500 [APP/PROC/WEB/0] OUT | 1474           | 47168       | Ljava/util/Hashtable$Entry;                                                                                      |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 30             | 31200       | [[C                                                                                                              |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 740            | 35520       | Lorg/springframework/util/ConcurrentReferenceHashMap$SoftEntryReference;                                         |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 927            | 29664       | Ljava/lang/invoke/MethodType$ConcurrentWeakInternSet$WeakEntry;                                                  |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 766            | 36768       | Lorg/apache/tomcat/util/modeler/AttributeInfo;                                                                   |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 1040           | 36440       | [Lorg/springframework/util/ConcurrentReferenceHashMap$Reference;                                                 |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 1125           | 36000       | Ljava/util/concurrent/locks/ReentrantLock$NonfairSync;                                                           |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 896            | 35840       | Ljava/lang/invoke/MethodType;                                                                                    |
   2018-01-23T08:23:03.47-0500 [APP/PROC/WEB/0] OUT | 1391           | 33384       | Ljava/util/ArrayList;                                                                                            |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 420            | 30240       | Lorg/springframework/core/annotation/AnnotationAttributes;                                                       |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 287            | 27552       | Lorg/springframework/beans/GenericTypeAwarePropertyDescriptor;                                                   |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 469            | 22512       | Ljava/util/concurrent/locks/ReentrantReadWriteLock$NonfairSync;                                                  |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 1040           | 24960       | Lorg/springframework/util/ConcurrentReferenceHashMap$ReferenceManager;                                           |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 287            | 20664       | Ljava/beans/PropertyDescriptor;                                                                                  |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 666            | 21312       | Ljava/lang/invoke/LambdaForm$Name;                                                                               |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 332            | 21248       | Ljava/util/concurrent/ConcurrentHashMap;                                                                         |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 679            | 21728       | Ljava/util/LinkedList;                                                                                           |
   2018-01-23T08:23:03.48-0500 [APP/PROC/WEB/0] OUT | 337            | 21632       | [Ljava/util/Hashtable$Entry;                                                                                     |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 120            | 20160       | Lorg/springframework/context/annotation/ConfigurationClassBeanDefinitionReader$ConfigurationClassBeanDefinition; |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 503            | 20120       | Lsun/util/locale/BaseLocale$Key;                                                                                 |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 314            | 20096       | [Ljava/lang/ref/SoftReference;                                                                                   |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 1236           | 19776       | Ljava/lang/ref/ReferenceQueue$Lock;                                                                              |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 382            | 18336       | Lorg/apache/tomcat/util/modeler/OperationInfo;                                                                   |
   2018-01-23T08:23:03.49-0500 [APP/PROC/WEB/0] OUT | 267            | 17088       | Lorg/springframework/core/MethodParameter;                                                                       |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 487            | 15584       | Ljava/lang/Package;                                                                                              |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 337            | 16176       | Ljava/util/Hashtable;                                                                                            |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 503            | 16096       | Lsun/util/locale/BaseLocale;                                                                                     |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 521            | 15616       | [Ljava/lang/CharSequence;                                                                                        |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 75             | 15312       | [Z                                                                                                               |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 334            | 13360       | Ljava/lang/Package$VersionInfo;                                                                                  |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 608            | 14592       | Ljava/util/jar/Attributes$Name;                                                                                  |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 272            | 13056       | Lch/qos/logback/classic/Logger;                                                                                  |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 516            | 12384       | Ljava/util/LinkedList$Node;                                                                                      |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 192            | 12288       | Ljava/net/URL;                                                                                                   |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 502            | 12048       | Ljava/util/Locale$LocaleKey;                                                                                     |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 190            | 15200       | Ljava/lang/reflect/Constructor;                                                                                  |
   2018-01-23T08:23:03.50-0500 [APP/PROC/WEB/0] OUT | 556            | 13344       | Ljava/lang/invoke/LambdaForm$NamedFunction;                                                                      |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 289            | 11560       | Ljava/lang/invoke/DirectMethodHandle$Accessor;                                                                   |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 693            | 11088       | Lorg/springframework/core/annotation/AnnotationUtils$DefaultValueHolder;                                         |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 470            | 11280       | Ljava/util/concurrent/locks/ReentrantReadWriteLock;                                                              |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 66             | 10560       | Lorg/springframework/beans/factory/support/RootBeanDefinition;                                                   |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 219            | 10512       | Lorg/springframework/core/ResolvableType;                                                                        |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 731            | 11696       | Ljava/util/LinkedHashSet;                                                                                        |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 218            | 10464       | Lcom/sun/beans/util/Cache$Kind$Soft;                                                                             |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 625            | 10000       | Ljava/lang/Integer;                                                                                              |
   2018-01-23T08:23:03.51-0500 [APP/PROC/WEB/0] OUT | 189            | 10584       | Ljava/lang/invoke/MethodTypeForm;                                                                                |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 165            | 9240        | Ljava/lang/invoke/LambdaForm;                                                                                    |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 248            | 9920        | Ljavax/management/MBeanOperationInfo;                                                                            |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 310            | 9920        | Ljavax/management/MBeanAttributeInfo;                                                                            |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 170            | 8912        | [Ljava/lang/invoke/LambdaForm$Name;                                                                              |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 409            | 9816        | Ljava/lang/module/ModuleDescriptor$Exports;                                                                      |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 104            | 8960        | [Ljava/util/WeakHashMap$Entry;                                                                                   |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 535            | 8560        | Ljava/util/LinkedHashMap$LinkedKeySet;                                                                           |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 261            | 8352        | Lorg/springframework/boot/context/properties/source/ConfigurationPropertyName;                                   |
   2018-01-23T08:23:03.52-0500 [APP/PROC/WEB/0] OUT | 285            | 9120        | Lorg/apache/tomcat/util/modeler/ParameterInfo;                                                                   |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 175            | 8400        | Ljava/util/logging/LogManager$LoggerWeakRef;                                                                     |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 245            | 7840        | Ljava/lang/invoke/DirectMethodHandle;                                                                            |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 318            | 7632        | Ljava/util/concurrent/CopyOnWriteArrayList;                                                                      |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 470            | 7520        | Ljava/util/concurrent/locks/ReentrantReadWriteLock$ReadLock;                                                     |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 470            | 7520        | Ljava/util/concurrent/locks/ReentrantReadWriteLock$WriteLock;                                                    |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 125            | 6824        | [Ljava/lang/invoke/MethodHandle;                                                                                 |
   2018-01-23T08:23:03.53-0500 [APP/PROC/WEB/0] OUT | 382            | 8024        | [Lorg/apache/tomcat/util/modeler/ParameterInfo;                                                                  |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 470            | 7520        | Ljava/util/concurrent/locks/ReentrantReadWriteLock$Sync$ThreadLocalHoldCounter;                                  |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 288            | 6912        | Ljava/lang/Long;                                                                                                 |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 418            | 6688        | Ljava/util/HashSet;                                                                                              |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 208            | 6656        | Ljavax/management/MBeanParameterInfo;                                                                            |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 264            | 6336        | Lorg/springframework/beans/BeanMetadataAttribute;                                                                |
   2018-01-23T08:23:03.54-0500 [APP/PROC/WEB/0] OUT | 264            | 6336        | Lorg/springframework/cglib/core/Signature;                                                                       |
   2018-01-23T08:23:03.55-0500 [APP/PROC/WEB/0] OUT | 259            | 6216        | Lorg/springframework/boot/context/properties/source/PropertyMapping;                                             |
   2018-01-23T08:23:03.55-0500 [APP/PROC/WEB/0] OUT | 360            | 5760        | Ljava/util/LinkedHashMap$LinkedEntrySet;                                                                         |
   2018-01-23T08:23:03.55-0500 [APP/PROC/WEB/0] OUT | 162            | 5184        | Lorg/springframework/asm/Type;                                                                                   |
   2018-01-23T08:23:03.55-0500 [APP/PROC/WEB/0] OUT | 192            | 6144        | Ljava/util/RegularEnumSet;                                                                                       |
   2018-01-23T08:23:03.55-0500 [APP/PROC/WEB/0] OUT | 65             | 5200        | [Lorg/springframework/util/ConcurrentReferenceHashMap$Segment;                                                   |
   2018-01-23T08:23:03.55-0500 [APP/PROC/WEB/0] OUT Memory usage:
   2018-01-23T08:23:03.56-0500 [APP/PROC/WEB/0] OUT    Non-heap memory: init 7667712, used 55847152, committed 59531264, max 1364553728
   2018-01-23T08:23:03.56-0500 [APP/PROC/WEB/0] OUT    Metaspace: init 0, used 37267616, committed 39153664, max 39153664
   2018-01-23T08:23:03.56-0500 [APP/PROC/WEB/0] OUT Memory pool usage:
   2018-01-23T08:23:03.55-0500 [APP/PROC/WEB/0] OUT    Heap memory: init 511705088, used 15251616, committed 51380224, max 511705088
   2018-01-23T08:23:03.56-0500 [APP/PROC/WEB/0] OUT    CodeHeap 'non-nmethods': init 2555904, used 1323776, committed 2555904, max 5836800
   2018-01-23T08:23:03.56-0500 [APP/PROC/WEB/0] OUT       Hint: Metaspace is over 95% full. To increase it, set -XX:MaxMetaspaceSize to a suitable value.
   2018-01-23T08:23:03.56-0500 [APP/PROC/WEB/0] OUT    CodeHeap 'profiled nmethods': init 2555904, used 10262144, committed 10289152, max 122908672
   2018-01-23T08:23:03.56-0500 [APP/PROC/WEB/0] OUT    Compressed Class Space: init 0, used 4509984, committed 4976640, max 1073741824
   2018-01-23T08:23:03.57-0500 [APP/PROC/WEB/0] OUT    G1 Eden Space: init 27262976, used 0, committed 29360128, max -1
   2018-01-23T08:23:03.57-0500 [APP/PROC/WEB/0] OUT    G1 Old Gen: init 484442112, used 15251616, committed 22020096, max 511705088
   2018-01-23T08:23:03.57-0500 [APP/PROC/WEB/0] ERR jvmkill killing current process
   2018-01-23T08:23:03.57-0500 [APP/PROC/WEB/0] OUT    G1 Survivor Space: init 0, used 0, committed 0, max -1
   2018-01-23T08:23:03.57-0500 [APP/PROC/WEB/0] OUT    CodeHeap 'non-profiled nmethods': init 2555904, used 2490624, committed 2555904, max 122912768
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT 2018-01-23 13:23:03
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT Full thread dump OpenJDK 64-Bit Server VM (9.0.4+11 mixed mode):
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.defineClass1(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.security.SecureClassLoader.defineClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader$1.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.security.AccessController.doPrivileged(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.loadClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:93)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.loadClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.defineClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader$1.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.security.AccessController.doPrivileged(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.loadClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:93)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.loadClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e25432d0> (a org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedContext)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e25432d0> (a org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedContext)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.FutureTask.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - parking to wait for  <0x00000000e2543770> (a java.util.concurrent.FutureTask)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.locks.LockSupport.park(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.FutureTask.awaitDone(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:939)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e25fec38> (a org.apache.catalina.core.StandardHost)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e25fec38> (a org.apache.catalina.core.StandardHost)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e25fec38> (a org.apache.catalina.core.StandardHost)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.FutureTask.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.Thread.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - waiting on <0x00000000e18bd0b0> (a java.lang.ref.ReferenceQueue$Lock)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at jdk.internal.ref.CleanerImpl.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.Thread.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at jdk.internal.misc.InnocuousThread.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "C1 CompilerThread3" #8 daemon prio=9 os_prio=0 tid=0x00007f03d426e800 nid=0x4d waiting on condition [0x0000000000000000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    No compile task
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    No compile task
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    Compiling: 4155       4       org.springframework.core.ResolvableType::as (81 bytes)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    No compile task
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f03d424d800 nid=0x48 in Object.wait() [0x00007f036d3fa000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - waiting on <0x00000000e18ccc78> (a java.lang.ref.ReferenceQueue$Lock)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.ReferenceQueue.remove(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.Reference.waitForReferencePendingList(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.Reference.processPendingReferences(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.Reference$ReferenceHandler.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.Reference.access$000(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at jdk.internal.misc.Unsafe.park(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.locks.LockSupport.park(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.FutureTask.get(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:939)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503530> (a org.apache.catalina.core.StandardEngine)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503530> (a org.apache.catalina.core.StandardEngine)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503530> (a org.apache.catalina.core.StandardEngine)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503530> (a org.apache.catalina.core.StandardEngine)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:98)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503a18> (a java.lang.Object)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.<init>(TomcatWebServer.java:84)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getTomcatWebServer(TomcatServletWebServerFactory.java:413)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getWebServer(TomcatServletWebServerFactory.java:176)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:150)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:138)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:751)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:387)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.run(SpringApplication.java:1245)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "VM Thread" os_prio=0 tid=0x00007f03d423f800 nid=0x46 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#2" os_prio=0 tid=0x00007f03d402c000 nid=0x34 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#5" os_prio=0 tid=0x00007f03d4031000 nid=0x37 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#7" os_prio=0 tid=0x00007f03d4034800 nid=0x39 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Main Marker" os_prio=0 tid=0x00007f03d4062000 nid=0x43 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#0" os_prio=0 tid=0x00007f03d4043000 nid=0x41 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#2" os_prio=0 tid=0x00007f03d403f800 nid=0x3f runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#3" os_prio=0 tid=0x00007f03d403e000 nid=0x3e runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#4" os_prio=0 tid=0x00007f03d403c000 nid=0x3d runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#5" os_prio=0 tid=0x00007f03d403a800 nid=0x3c runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#7" os_prio=0 tid=0x00007f03d4037000 nid=0x3a runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "localhost-startStop-1" #16 daemon prio=5 os_prio=0 tid=0x00007f0330b88000 nid=0x70 runnable [0x00007f032bbf9000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.defineClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader.defineClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader.access$100(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader.findClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader$1.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e268e8c8> (a java.lang.Object)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ClassLoader.defineClass1(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.security.SecureClassLoader.defineClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader.access$100(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader$1.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader.findClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e268a6a0> (a java.lang.Object)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.tomcat.websocket.server.WsSci.init(WsSci.java:131)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.tomcat.websocket.server.WsContextListener.contextInitialized(WsContextListener.java:39)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.net.URLClassLoader.defineClass(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.Thread.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "Tomcat-startStop-1" #15 daemon prio=5 os_prio=0 tid=0x00007f03d46cf000 nid=0x6f waiting on condition [0x00007f036c1c6000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at jdk.internal.misc.Unsafe.park(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: WAITING (parking)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.FutureTask.get(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:872)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "Service Thread" #11 daemon prio=9 os_prio=0 tid=0x00007f03d4320000 nid=0x50 runnable [0x0000000000000000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "Common-Cleaner" #10 daemon prio=8 os_prio=0 tid=0x00007f03d42a5000 nid=0x4f in Object.wait() [0x00007f036c8c9000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.Object.wait(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: TIMED_WAITING (on object monitor)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "Sweeper thread" #9 daemon prio=9 os_prio=0 tid=0x00007f03d4270800 nid=0x4e runnable [0x0000000000000000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.ReferenceQueue.remove(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - waiting to re-lock in wait() <0x00000000e18bd0b0> (a java.lang.ref.ReferenceQueue$Lock)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "C2 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f03d426a800 nid=0x4b runnable [0x0000000000000000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "C2 CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007f03d426c800 nid=0x4c waiting on condition [0x0000000000000000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f03d4267000 nid=0x49 waiting on condition [0x0000000000000000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f03d4268800 nid=0x4a waiting on condition [0x0000000000000000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: RUNNABLE
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: WAITING (on object monitor)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.Finalizer$FinalizerThread.run(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.Object.wait(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.ref.ReferenceQueue.remove(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - waiting to re-lock in wait() <0x00000000e18ccc78> (a java.lang.ref.ReferenceQueue$Lock)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f03d4247800 nid=0x47 waiting on condition [0x00007f036d4fb000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "main" #1 prio=5 os_prio=0 tid=0x00007f03d4013800 nid=0x30 waiting on condition [0x00007f03db63a000]
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.util.concurrent.FutureTask.awaitDone(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT    java.lang.Thread.State: WAITING (parking)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - parking to wait for  <0x00000000e25034f8> (a java.util.concurrent.FutureTask)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.StandardService.startInternal(StandardService.java:422)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503900> (a org.apache.catalina.core.StandardServer)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.startup.Tomcat.start(Tomcat.java:367)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503830> (a org.apache.catalina.core.StandardService)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:793)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e2503878> (a java.lang.Object)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:177)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.run(SpringApplication.java:327)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     - locked <0x00000000e18cd190> (a java.lang.Object)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at com.mycompany.connecttimeouttest.ConnectTimeoutTestApplication.main(ConnectTimeoutTestApplication.java:10)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.SpringApplication.run(SpringApplication.java:1233)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9.0.4/Native Method)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at java.lang.reflect.Method.invoke(java.base@9.0.4/Unknown Source)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#0" os_prio=0 tid=0x00007f03d4029000 nid=0x32 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#1" os_prio=0 tid=0x00007f03d402a800 nid=0x33 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT     at org.springframework.boot.loader.WarLauncher.main(WarLauncher.java:58)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#3" os_prio=0 tid=0x00007f03d402e000 nid=0x35 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#4" os_prio=0 tid=0x00007f03d402f800 nid=0x36 runnable 
   2018-01-23T08:23:03.60-0500 [APP/PROC/WEB/0] OUT Heap
   2018-01-23T08:23:03.60-0500 [APP/PROC/WEB/0] OUT   region size 1024K, 1 young (1024K), 0 survivors (0K)
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Marker#1" os_prio=0 tid=0x00007f03d4065800 nid=0x45 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#6" os_prio=0 tid=0x00007f03d4038800 nid=0x3b runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "VM Periodic Task Thread" os_prio=0 tid=0x00007f03d4322800 nid=0x51 waiting on condition 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "GC Thread#6" os_prio=0 tid=0x00007f03d4033000 nid=0x38 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Marker#0" os_prio=0 tid=0x00007f03d4064000 nid=0x44 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Refine#1" os_prio=0 tid=0x00007f03d4041800 nid=0x40 runnable 
   2018-01-23T08:23:03.58-0500 [APP/PROC/WEB/0] OUT "G1 Young RemSet Sampling" os_prio=0 tid=0x00007f03d4045000 nid=0x42 runnable 
   2018-01-23T08:23:03.60-0500 [APP/PROC/WEB/0] OUT JNI global references: 226810
   2018-01-23T08:23:03.60-0500 [APP/PROC/WEB/0] OUT  garbage-first heap   total 50176K, used 14894K [0x00000000e1800000, 0x00000000e1900188, 0x0000000100000000)
   2018-01-23T08:23:03.60-0500 [APP/PROC/WEB/0] OUT  Metaspace       used 36394K, capacity 37422K, committed 38236K, reserved 1083392K
   2018-01-23T08:23:03.60-0500 [APP/PROC/WEB/0] OUT   class space    used 4404K, capacity 4623K, committed 4860K, reserved 1048576K
   2018-01-23T08:23:08.57-0500 [APP/PROC/WEB/0] ERR jvmkill killing current process
   2018-01-23T08:23:08.60-0500 [APP/PROC/WEB/0] OUT Exit status 137
   2018-01-23T08:23:08.61-0500 [CELL/0] OUT Exit status 0
   2018-01-23T08:23:08.68-0500 [CELL/0] OUT Destroying container
   2018-01-23T08:23:08.70-0500 [API/1] OUT Process has crashed with type: "web"
@nebhale
Copy link
Member

nebhale commented Jan 23, 2018

The memory space output printed after the OOM gives you an indication of the issue:

OUT    Metaspace: init 0, used 37267616, committed 39153664, max 39153664
OUT       Hint: Metaspace is over 95% full. To increase it, set -XX:MaxMetaspaceSize to a suitable value.

(Note that the order is a bit screwed up. Updating the latest version of the client, and the latest version of Loggergator/CF sorts this out.)

The calculated memory configuration is listed as the first line of output:

OUT JVM Memory Configuration: -Xss1M -Xmx498339K -XX:MaxMetaspaceSize=38236K -XX:ReservedCodeCacheSize=240M -XX:MaxDirectMemorySize=10M

So it would appear that the calculator is choosing 38M which is way too small to be functional. I just pushed a simple Spring Boot application against Java 8 and it chose ~100M. That seems to indicate that it's not noticing all of the possible Java 9 classes (guessing that they've moved rt.jar or it's equivalent) and doing the calculation incorrectly. I'd consider this a bug/Java 9 improvement.

@nebhale nebhale self-assigned this Jan 23, 2018
@nebhale nebhale added this to the v4.8 milestone Jan 23, 2018
nebhale added a commit that referenced this issue Jan 23, 2018
@nebhale
Java 9 Class Count
42521cd 
Previously, when counting classes, the buildpack only looked into JAR files
for classes that might be loaded.  In Java 9, the filenames have changed
(while maintaining their "ZIP-ness"), and now have .jmod extensions.  This
change updates the class count algorithm to examine .jmod files as well.

[resolves #543]
@edeandrea
Copy link
Author

Thanks @nebhale I meant to come back & post some more information as I've been debugging this morning on my side. When pushing to PCF if I add JAVA_OPTS: '-XX:MaxMetaspaceSize=64m' to my manifest then my app is fine.

@nebhale
Copy link
Member

nebhale commented Jan 23, 2018 •
edited

As expected 😄. I'm working on it now, but this new binary format doesn't seem too examinable.

@nebhale nebhale modified the milestones: v4.8, v4.9 Feb 12, 2018
@mrkk
Copy link

mrkk commented Jul 1, 2019

do you have to do something special to get the hints printed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nebhale nebhale

Labels
bug jre
Projects
None yet
Milestone
v4.9
Development

No branches or pull requests
3 participants
@nebhale @edeandrea @mrkk