5 Replies Latest reply on Jul 6, 2018 5:22 PM by cviniciusm

    Wildfly 12 datasource JNDI lookup returning null

    bowlerma

      We've recently migrated to Wildfly 12 from Wildfly 10.1 and have started to see intermittent failures during startup due lookup of the datasource via JNDI returning null.

       

      The datasource is configured within the standalone.xml file and looks like the following:

       

                      
                          ${env.CONN_URL}
                          sqlserver
                          TRANSACTION_READ_COMMITTED
                          
                              0
                              0
                              50
                              false
                              false
                          
                          
                              encrypted-realm
                          
                          
                              select 1
                              true
                          
                          
                              60000
                              1
                              12
                              5000
                          
                      
                      
                          
                              com.microsoft.sqlserver.jdbc.SQLServerDriver
                          
                      

       

      We are using the old Picketbox module for encrypting the user/password.  We are also using JBeret configured to store configuration within the ControlDB database.

       

      We have the following defined with the persistence.xml file:

       

      org.hibernate.jpa.HibernatePersistenceProvider
      java:jboss/datasources/ControlDB
      
      
      

       

      We're using multi-tenancy and have a class that extends AbstractMultiTenantConnectionProvider whose constructor looks something like the following:

       

      public class MultiTenantConnectionProvider extends AbstractMultiTenantConnectionProvider {
      
      
          public MultiTenantConnectionProvider() {
              DataSource ds = null;
              try {
                  Context ctx = new InitialContext();
                  ds = (DataSource) ctx.lookup("java:jboss/datasources/ControlDB");
              catch (NamingException e) {
                  LOG.error("Naming Exception: ", e);
              }
      
      
              if (ds == null) {
                  throw new IllegalArgumentException("No Control Database could be found");
              }
          }

       

      Every now and then Wildfly doesn't startup, it doesn't fail but appears to hang and we see the following exception in the logs:

       

      2018-06-20 13:11:09,918     WARN  [org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator] (ServerService Thread Pool -- 58)  Unable to instantiate specified class [com.test.multitenancy.MultiTenantConnectionProvider]: java.lang.IllegalArgumentException: No Control Database could be found
      at com.test.MultiTenantConnectionProvider.(MultiTenantConnectionProvider.java:61)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at java.lang.Class.newInstance(Class.java:442)
      at org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator.initiateService(MultiTenantConnectionProviderInitiator.java:83)
      at org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator.initiateService(MultiTenantConnectionProviderInitiator.java:28)
      at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:254)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:228)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)
      at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.buildJdbcConnectionAccess(JdbcEnvironmentInitiator.java:149)
      at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:66)
      at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
      at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:254)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:228)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)
      at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:51)
      at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:94)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
      at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)
      at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.handleTypes(MetadataBuildingProcess.java:352)
      at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:111)
      at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:848)
      at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:875)
      at org.jboss.as.jpa.hibernate5.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44)
      at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:167)
      at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:125)
      at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:650)
      at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:209)
      at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
      at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
      at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
      at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
      at java.lang.Thread.run(Thread.java:748)
      at org.jboss.threads.JBossThread.run(JBossThread.java:485)

       

      I've seen some old posts that imply that a resource-ref for the JNDI binding also needs to be created in a deployment descriptor, but am not sure if this is relevant, and why we would need to do this now since upgrading to WF12.

       

      Any ideas?

       

      Regards,

       

      Mark

        • 1. Re: Wildfly 12 datasource JNDI lookup returning null
          mchoma

          Interesting. Can you try with WF 13, please, if it is still present? Maybe it is some race condition in registering datasource vs. jpa subsystems - just guessing. Also can you post again/edit your standalone.xml and persistence.xml snippet. I don't see xml tags for some reason.

           

          And when you put resource-ref for the JNDI binding into deployment descriptor does problem still occure?

          • 2. Re: Wildfly 12 datasource JNDI lookup returning null
            bowlerma

            Oops, not sure what happened there.

             

            standalone.xml

             

                    <subsystem xmlns="urn:jboss:domain:datasources:5.0">

                        <datasources>

                            <datasource jta="true" jndi-name="java:jboss/datasources/ControlDB" pool-name="ControlDB" enabled="true" use-java-context="true" statistics-enabled="true">

                                <connection-url>${env.CONN_URL}</connection-url>

                                <driver>sqlserver</driver>

                                <transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>

                                <pool>

                                    <min-pool-size>0</min-pool-size>

                                    <initial-pool-size>0</initial-pool-size>

                                    <max-pool-size>50</max-pool-size>

                                    <prefill>false</prefill>

                                    <use-strict-min>false</use-strict-min>

                                </pool>

                                <security>

                                    <security-domain>encrypted-realm</security-domain>

                                </security>

                                <validation>

                                    <check-valid-connection-sql>select 1</check-valid-connection-sql>

                                    <validate-on-match>true</validate-on-match>

                                </validation>

                                <timeout>

                                    <blocking-timeout-millis>60000</blocking-timeout-millis>

                                    <idle-timeout-minutes>1</idle-timeout-minutes>

                                    <allocation-retry>12</allocation-retry>

                                    <allocation-retry-wait-millis>5000</allocation-retry-wait-millis>

                                </timeout>

                            </datasource>

                            <drivers>

                                <driver name="sqlserver" module="com.sqlserver">

                                    <driver-class>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver-class>

                                </driver>

                            </drivers>

                        </datasources>

                    </subsystem>

             

            persistence.xml (I've removed the class list)

             

            <persistence-unit name="controlDB" transaction-type="JTA">

                <provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>

                <jta-data-source>java:jboss/datasources/ControlDB</jta-data-source>

            </persistence-unit>

             

            I did try adding a reference to the web.xml deployment descriptor, as follows:

             

              <resource-env-ref>

                <resource-env-ref-name>datasources/ControlDB</resource-env-ref-name>

                <resource-env-ref-type>javax.sql.DataSource</resource-env-ref-type>

                <lookup-name>java:jboss/datasources/ControlDB</lookup-name>

              </resource-env-ref>

             

            That gave me a slightly different error:

             

            2018-06-20 16:38:50,271     ERROR [com.test.multitenancy.MultiTenantConnectionProvider] (ServerService Thread Pool -- 55)  Naming Exception: : javax.naming.NameNotFoundException: env/datasources/ControlDB [Root exception is java.lang.IllegalStateException]

            at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:153)

            at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:83)

            at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:207)

            at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

            at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:239)

            at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:193)

            at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:189)

            at javax.naming.InitialContext.lookup(InitialContext.java:417)

            at javax.naming.InitialContext.lookup(InitialContext.java:417)

            at com.test.multitenancy.MultiTenantConnectionProvider.jndiLookupControlDatabase(MultiTenantConnectionProvider.java:123)

            at com.test.multitenancy.MultiTenantConnectionProvider.<init>(MultiTenantConnectionProvider.java:59)

            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

            at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

            at java.lang.Class.newInstance(Class.java:442)

            at org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator.initiateService(MultiTenantConnectionProviderInitiator.java:83)

            at org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator.initiateService(MultiTenantConnectionProviderInitiator.java:28)

            at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:254)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:228)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)

            at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.buildJdbcConnectionAccess(JdbcEnvironmentInitiator.java:149)

            at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:66)

            at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)

            at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:254)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:228)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)

            at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:51)

            at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:94)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)

            at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)

            at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.handleTypes(MetadataBuildingProcess.java:352)

            at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:111)

            at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:848)

            at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:875)

            at org.jboss.as.jpa.hibernate5.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44)

            at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:167)

            at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:125)

            at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:650)

            at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:209)

            at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)

            at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)

            at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)

            at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)

            at java.lang.Thread.run(Thread.java:748)

            at org.jboss.threads.JBossThread.run(JBossThread.java:485)

            Caused by: java.lang.IllegalStateException

            at org.jboss.msc.value.InjectedValue.getValue(InjectedValue.java:47)

            at org.jboss.as.naming.service.BinderService.getValue(BinderService.java:148)

            at org.jboss.as.naming.service.BinderService.getValue(BinderService.java:46)

            at org.jboss.msc.service.ServiceControllerImpl.getValue(ServiceControllerImpl.java:1095)

            at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)

            ... 47 more

             

            The application is running in a container in the Azure cloud.  I've not seen the problem when the container is run locally.  Also, whenever I've increased the logging level to try to get more detail, I've not been able to recreate the issue, which possibly does hint at a race condition as you have suggested.

             

            Cheers,

             

            Mark

            • 3. Re: Wildfly 12 datasource JNDI lookup returning null
              simkam

              Can you post server.log from failed start?

              • 4. Re: Wildfly 12 datasource JNDI lookup returning null
                bowlerma

                Sorry for delay in responding.

                 

                I should mention that we are using the deployment scanner to deploy our war files on startup.  The deployment scanner is configured as below:

                 

                        <subsystem xmlns="urn:jboss:domain:deployment-scanner:2.0">

                <deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="0" deployment-timeout="900" auto-deploy-exploded="false" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}"/>

                        </subsystem>

                 

                Log output is below.  Log output is from the boot log which has a log level of INFO and the server log (configured in standalone) which has a log level of WARN.

                 

                2018-07-03 09:05:22,325 INFO  [org.jboss.modules] (main) JBoss Modules version 1.7.0.Final
                2018-07-03 09:05:23,243 INFO  [org.jboss.msc] (main) JBoss MSC version 1.3.2.Final
                2018-07-03 09:05:23,280 INFO  [org.jboss.threads] (main) JBoss Threads version 2.3.1.Final
                2018-07-03 09:05:23,745 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049: WildFly Full 12.0.0.Final (WildFly Core 4.0.0.Final) starting
                2018-07-03 09:05:23,748 DEBUG [org.jboss.as.config] (MSC service thread 1-2) Configured system properties:
                [Standalone] =
                awt.toolkit = sun.awt.X11.XToolkit
                file.encoding = ANSI_X3.4-1968
                file.encoding.pkg = sun.io
                file.separator = /
                java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
                java.awt.headless = true
                java.awt.printerjob = sun.print.PSPrinterJob
                java.class.path = /opt/jboss/wildfly/jboss-modules.jar
                java.class.version = 52.0
                java.endorsed.dirs = /usr/lib/jvm/zulu-8-amd64/jre/lib/endorsed
                java.ext.dirs = /usr/lib/jvm/zulu-8-amd64/jre/lib/ext:/usr/java/packages/lib/ext
                java.home = /usr/lib/jvm/zulu-8-amd64/jre
                java.io.tmpdir = /tmp
                java.library.path = /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
                java.net.preferIPv4Stack = true
                java.runtime.name = OpenJDK Runtime Environment
                java.runtime.version = 1.8.0_144-b01
                java.specification.name = Java Platform API Specification
                java.specification.vendor = Oracle Corporation
                java.specification.version = 1.8
                java.util.logging.manager = org.jboss.logmanager.LogManager
                java.vendor = Azul Systems, Inc.
                java.vendor.url = http://www.azulsystems.com/
                java.vendor.url.bug = http://www.azulsystems.com/support/
                java.version = 1.8.0_144
                java.vm.info = mixed mode
                java.vm.name = OpenJDK 64-Bit Server VM
                java.vm.specification.name = Java Virtual Machine Specification
                java.vm.specification.vendor = Oracle Corporation
                java.vm.specification.version = 1.8
                java.vm.vendor = Azul Systems, Inc.
                java.vm.version = 25.144-b01
                javax.management.builder.initial = org.jboss.as.jmx.PluggableMBeanServerBuilder
                javax.xml.datatype.DatatypeFactory = __redirected.__DatatypeFactory
                javax.xml.parsers.DocumentBuilderFactory = __redirected.__DocumentBuilderFactory
                javax.xml.parsers.SAXParserFactory = __redirected.__SAXParserFactory
                javax.xml.stream.XMLEventFactory = __redirected.__XMLEventFactory
                javax.xml.stream.XMLInputFactory = __redirected.__XMLInputFactory
                javax.xml.stream.XMLOutputFactory = __redirected.__XMLOutputFactory
                javax.xml.transform.TransformerFactory = __redirected.__TransformerFactory
                javax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema = __redirected.__SchemaFactory
                javax.xml.xpath.XPathFactory:http://java.sun.com/jaxp/xpath/dom = __redirected.__XPathFactory
                jboss.bind.address = 0.0.0.0
                jboss.home.dir = /opt/jboss/wildfly
                jboss.host.name = 7a30ca5e653d
                jboss.modules.dir = /opt/jboss/wildfly/modules
                jboss.modules.system.pkgs = org.jboss.byteman
                jboss.node.name = 7a30ca5e653d
                jboss.qualified.host.name = 7a30ca5e653d
                jboss.server.base.dir = /opt/jboss/wildfly/standalone
                jboss.server.config.dir = /opt/jboss/wildfly/standalone/configuration
                jboss.server.data.dir = /opt/jboss/wildfly/standalone/data
                jboss.server.deploy.dir = /opt/jboss/wildfly/standalone/data/content
                jboss.server.log.dir = /home/LogFiles/2018-07-03
                jboss.server.name = 7a30ca5e653d
                jboss.server.persist.config = true
                jboss.server.temp.dir = /opt/jboss/wildfly/standalone/tmp
                line.separator =
                logging.configuration = file:/opt/jboss/wildfly/standalone/configuration/logging.properties
                module.path = /opt/jboss/wildfly/modules
                org.jboss.boot.log.file = /home/LogFiles/2018-07-03/server.log
                org.jboss.resolver.warning = true
                org.xml.sax.driver = __redirected.__XMLReaderFactory
                os.arch = amd64
                os.name = Linux
                os.version = 4.4.0-119-generic
                path.separator = :
                sun.arch.data.model = 64
                sun.boot.class.path = /usr/lib/jvm/zulu-8-amd64/jre/lib/resources.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/rt.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/jsse.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/jce.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/charsets.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/jfr.jar:/usr/lib/jvm/zulu-8-amd64/jre/classes
                sun.boot.library.path = /usr/lib/jvm/zulu-8-amd64/jre/lib/amd64
                sun.cpu.endian = little
                sun.cpu.isalist =
                sun.io.unicode.encoding = UnicodeLittle
                sun.java.command = /opt/jboss/wildfly/jboss-modules.jar -mp /opt/jboss/wildfly/modules org.jboss.as.standalone -Djboss.home.dir=/opt/jboss/wildfly -Djboss.server.base.dir=/opt/jboss/wildfly/standalone -Djboss.server.log.dir=/home/LogFiles/2018-07-03 -b 0.0.0.0
                sun.java.launcher = SUN_STANDARD
                sun.jnu.encoding = ANSI_X3.4-1968
                sun.management.compiler = HotSpot 64-Bit Tiered Compilers
                sun.os.patch.level = unknown
                user.country = US
                user.dir = /
                user.home = /root
                user.language = en
                user.name = root
                user.timezone = Etc/UTC
                2018-07-03 09:05:23,749 DEBUG [org.jboss.as.config] (MSC service thread 1-2) VM Arguments: -D[Standalone] -Xms512M -Xmx1792M -Djava.net.preferIPv4Stack=true -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/LogFiles/GCLogs/ -XX:+PrintGCDetails -XX:+PrintGCDateStamps -verbose:gc -Xloggc:/home/LogFiles/GCLogs/7a30ca5e653d_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=50M -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Dorg.jboss.boot.log.file=/home/LogFiles/2018-07-03/server.log -Dlogging.configuration=file:/opt/jboss/wildfly/standalone/configuration/logging.properties
                2018-07-03 09:05:28,077 INFO  [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
                2018-07-03 09:05:28,235 INFO  [org.wildfly.security] (ServerService Thread Pool -- 4) ELY00001: WildFly Elytron version 1.2.2.Final
                2018-07-03 09:05:29,078 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/b9/f1273098288e8eaf6a3a01eba3b40dbd6eb1e1/content
                2018-07-03 09:05:29,827 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/cf/01ba8168af80d7e12f269b14ffdaa582025047/content
                2018-07-03 09:05:31,122 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/5c/cd90320854fa469bc775a4cd0cb90cbdd17c3d/content
                2018-07-03 09:05:32,547 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/15/cdd8daaa416c274367883125deb34ace0a6c5b/content
                2018-07-03 09:05:33,956 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/dd/43f9e5744f41362bc790818371b5171bfd87e3/content
                2018-07-03 09:05:35,337 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/60/ee247f750e6bd1b44a0da938227cef2ff4a7cc/content
                2018-07-03 09:05:36,885 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/7d/436b15f14049d5987179289033f3728bfa5183/content
                2018-07-03 09:05:37,713 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/a7/9d0cceaafbbafea10ed7e75489cd0683b349c7/content
                2018-07-03 09:05:39,602 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/d0/c814168528deb68f4176d9f398bff88afc7d34/content
                2018-07-03 09:05:40,678 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/b7/46eb8e8659999f7645b56ffcd466a2c202636b/content
                2018-07-03 09:05:42,335 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/6a/3514b945647bfaa019562d72b5a941ec76de6a/content
                2018-07-03 09:05:44,484 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 16) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/fd/92b0e6e92312bc30a56dc46ac2cb40e3e565ee/content
                2018-07-03 09:05:44,546 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
                2018-07-03 09:05:45,309 INFO  [org.xnio] (MSC service thread 1-4) XNIO version 3.6.2.Final
                2018-07-03 09:05:45,346 INFO  [org.xnio.nio] (MSC service thread 1-4) XNIO NIO Implementation Version 3.6.2.Final
                2018-07-03 09:05:45,484 INFO  [org.jboss.remoting] (MSC service thread 1-6) JBoss Remoting version 5.0.5.Final
                2018-07-03 09:05:45,609 WARN  [org.jboss.as.logging] (Controller Boot Thread) WFLYLOG0012: Replacing handler 'FILE' during add operation. Either the handler type or the module name differs from the initial configuration.
                
                
                2018-07-03 09:05:47,668    WARN  [org.jboss.as.domain.management.security] (MSC service thread 1-3)  WFLYDM0111: Keystore /opt/jboss/wildfly/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
                2018-07-03 09:05:47,795    ERROR [stderr] (MSC service thread 1-8)  SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
                2018-07-03 09:06:20,129    WARN  [org.jboss.as.server.deployment] (MSC service thread 1-2)  WFLYSRV0059: Class Path entry xercesImpl.jar in /content/redacted.war/WEB-INF/lib/xalan-2.7.0.jar  does not point to a valid jar for a Class-Path reference.
                2018-07-03 09:06:22,543    WARN  [org.jboss.as.ejb3.deployment] (MSC service thread 1-7)  WFLYEJB0119: [EJB3.1 spec, section 4.9.2] Session bean implementation class MUST be public, not abstract and not final - uk.co.test.OAuthServiceProducer won't be considered as a session bean, since it doesn't meet that requirement
                2018-07-03 09:06:34,673    WARN  [org.jboss.as.jaxrs] (MSC service thread 1-4)  WFLYRS0015: No Servlet declaration found for JAX-RS application.  In redacted.war either provide a class that extends javax.ws.rs.core.Application or declare a servlet class in web.xml.
                2018-07-03 09:06:51,260    WARN  [org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator] (ServerService Thread Pool -- 55)  Unable to instantiate specified class [com.test.MultiTenantConnectionProvider]: java.lang.IllegalArgumentException: No Control Database could be found
                at com.test.MultiTenantConnectionProvider.(MultiTenantConnectionProvider.java:61)
                at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
                at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
                at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
                at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
                at java.lang.Class.newInstance(Class.java:442)
                at org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator.initiateService(MultiTenantConnectionProviderInitiator.java:83)
                at org.hibernate.engine.jdbc.connections.internal.MultiTenantConnectionProviderInitiator.initiateService(MultiTenantConnectionProviderInitiator.java:28)
                at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:254)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:228)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)
                at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.buildJdbcConnectionAccess(JdbcEnvironmentInitiator.java:149)
                at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:66)
                at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
                at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:88)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:254)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:228)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)
                at org.hibernate.engine.jdbc.internal.JdbcServicesImpl.configure(JdbcServicesImpl.java:51)
                at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.configureService(StandardServiceRegistryImpl.java:94)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
                at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:207)
                at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.handleTypes(MetadataBuildingProcess.java:352)
                at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:111)
                at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:848)
                at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:875)
                at org.jboss.as.jpa.hibernate5.TwoPhaseBootstrapImpl.build(TwoPhaseBootstrapImpl.java:44)
                at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:167)
                at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1$1.run(PersistenceUnitServiceImpl.java:125)
                at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:650)
                at org.jboss.as.jpa.service.PersistenceUnitServiceImpl$1.run(PersistenceUnitServiceImpl.java:209)
                at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
                at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
                at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378)
                at java.lang.Thread.run(Thread.java:748)
                at org.jboss.threads.JBossThread.run(JBossThread.java:485)

                 

                I've tried changing our startup process so that the wars are no longer deployed on startup, but are deployed after Wildfly has been started.  Tried this using both the deployment scanner and the management CLI.  This did the resolve the issue with the data source, however the application would still occasionally hang on startup (CPU time on the standalone.sh/java process never increased and the process did not exit).  Log files from this are below:

                 

                2018-07-03 09:54:57,569 INFO  [org.jboss.modules] (main) JBoss Modules version 1.7.0.Final
                2018-07-03 09:55:07,515 INFO  [org.jboss.msc] (main) JBoss MSC version 1.3.2.Final
                2018-07-03 09:55:07,576 INFO  [org.jboss.threads] (main) JBoss Threads version 2.3.1.Final
                2018-07-03 09:55:11,137 INFO  [org.jboss.as] (MSC service thread 1-1) WFLYSRV0049: WildFly Full 12.0.0.Final (WildFly Core 4.0.0.Final) starting
                2018-07-03 09:55:11,150 DEBUG [org.jboss.as.config] (MSC service thread 1-1) Configured system properties:
                [Standalone] =
                awt.toolkit = sun.awt.X11.XToolkit
                file.encoding = ANSI_X3.4-1968
                file.encoding.pkg = sun.io
                file.separator = /
                java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment
                java.awt.headless = true
                java.awt.printerjob = sun.print.PSPrinterJob
                java.class.path = /opt/jboss/wildfly/jboss-modules.jar
                java.class.version = 52.0
                java.endorsed.dirs = /usr/lib/jvm/zulu-8-amd64/jre/lib/endorsed
                java.ext.dirs = /usr/lib/jvm/zulu-8-amd64/jre/lib/ext:/usr/java/packages/lib/ext
                java.home = /usr/lib/jvm/zulu-8-amd64/jre
                java.io.tmpdir = /tmp
                java.library.path = /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
                java.net.preferIPv4Stack = true
                java.runtime.name = OpenJDK Runtime Environment
                java.runtime.version = 1.8.0_144-b01
                java.specification.name = Java Platform API Specification
                java.specification.vendor = Oracle Corporation
                java.specification.version = 1.8
                java.util.logging.manager = org.jboss.logmanager.LogManager
                java.vendor = Azul Systems, Inc.
                java.vendor.url = http://www.azulsystems.com/
                java.vendor.url.bug = http://www.azulsystems.com/support/
                java.version = 1.8.0_144
                java.vm.info = mixed mode
                java.vm.name = OpenJDK 64-Bit Server VM
                java.vm.specification.name = Java Virtual Machine Specification
                java.vm.specification.vendor = Oracle Corporation
                java.vm.specification.version = 1.8
                java.vm.vendor = Azul Systems, Inc.
                java.vm.version = 25.144-b01
                javax.management.builder.initial = org.jboss.as.jmx.PluggableMBeanServerBuilder
                javax.xml.datatype.DatatypeFactory = __redirected.__DatatypeFactory
                javax.xml.parsers.DocumentBuilderFactory = __redirected.__DocumentBuilderFactory
                javax.xml.parsers.SAXParserFactory = __redirected.__SAXParserFactory
                javax.xml.stream.XMLEventFactory = __redirected.__XMLEventFactory
                javax.xml.stream.XMLInputFactory = __redirected.__XMLInputFactory
                javax.xml.stream.XMLOutputFactory = __redirected.__XMLOutputFactory
                javax.xml.transform.TransformerFactory = __redirected.__TransformerFactory
                javax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema = __redirected.__SchemaFactory
                javax.xml.xpath.XPathFactory:http://java.sun.com/jaxp/xpath/dom = __redirected.__XPathFactory
                jboss.bind.address = 0.0.0.0
                jboss.home.dir = /opt/jboss/wildfly
                jboss.host.name = 0d7813512e96
                jboss.modules.dir = /opt/jboss/wildfly/modules
                jboss.modules.system.pkgs = org.jboss.byteman
                jboss.node.name = 0d7813512e96
                jboss.qualified.host.name = 0d7813512e96
                jboss.server.base.dir = /opt/jboss/wildfly/standalone
                jboss.server.config.dir = /opt/jboss/wildfly/standalone/configuration
                jboss.server.data.dir = /opt/jboss/wildfly/standalone/data
                jboss.server.deploy.dir = /opt/jboss/wildfly/standalone/data/content
                jboss.server.log.dir = /home/LogFiles/2018-07-03
                jboss.server.name = 0d7813512e96
                jboss.server.persist.config = true
                jboss.server.temp.dir = /opt/jboss/wildfly/standalone/tmp
                line.separator =
                logging.configuration = file:/opt/jboss/wildfly/standalone/configuration/logging.properties
                module.path = /opt/jboss/wildfly/modules
                org.jboss.boot.log.file = /home/LogFiles/2018-07-03/server.log
                org.jboss.resolver.warning = true
                org.xml.sax.driver = __redirected.__XMLReaderFactory
                os.arch = amd64
                os.name = Linux
                os.version = 4.4.0-119-generic
                path.separator = :
                sun.arch.data.model = 64
                sun.boot.class.path = /usr/lib/jvm/zulu-8-amd64/jre/lib/resources.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/rt.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/sunrsasign.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/jsse.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/jce.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/charsets.jar:/usr/lib/jvm/zulu-8-amd64/jre/lib/jfr.jar:/usr/lib/jvm/zulu-8-amd64/jre/classes
                sun.boot.library.path = /usr/lib/jvm/zulu-8-amd64/jre/lib/amd64
                sun.cpu.endian = little
                sun.cpu.isalist =
                sun.io.unicode.encoding = UnicodeLittle
                sun.java.command = /opt/jboss/wildfly/jboss-modules.jar -mp /opt/jboss/wildfly/modules org.jboss.as.standalone -Djboss.home.dir=/opt/jboss/wildfly -Djboss.server.base.dir=/opt/jboss/wildfly/standalone -Djboss.server.log.dir=/home/LogFiles/2018-07-03 -b 0.0.0.0
                sun.java.launcher = SUN_STANDARD
                sun.jnu.encoding = ANSI_X3.4-1968
                sun.management.compiler = HotSpot 64-Bit Tiered Compilers
                sun.os.patch.level = unknown
                user.country = US
                user.dir = /
                user.home = /root
                user.language = en
                user.name = root
                user.timezone = Etc/UTC
                2018-07-03 09:55:11,151 DEBUG [org.jboss.as.config] (MSC service thread 1-1) VM Arguments: -D[Standalone] -Xms512M -Xmx2048M -Djava.net.preferIPv4Stack=true -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/LogFiles/GCLogs/ -XX:+PrintGCDetails -XX:+PrintGCDateStamps -verbose:gc -Xloggc:/home/LogFiles/GCLogs/0d7813512e96_gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=50M -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Dorg.jboss.boot.log.file=/home/LogFiles/2018-07-03/server.log -Dlogging.configuration=file:/opt/jboss/wildfly/standalone/configuration/logging.properties
                2018-07-03 09:55:52,144 INFO  [org.jboss.as.controller.management-deprecated] (Controller Boot Thread) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/core-service=management/management-interface=http-interface' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation.
                2018-07-03 09:55:53,102 INFO  [org.wildfly.security] (ServerService Thread Pool -- 24) ELY00001: WildFly Elytron version 1.2.2.Final
                2018-07-03 09:55:53,517 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,563 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,564 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,572 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,581 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,588 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,600 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,610 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,619 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,624 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,647 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:55:53,654 INFO  [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) WFLYDS0004: Found redacted.war in deployment directory. To trigger deployment create a file called redacted.war.dodeploy
                2018-07-03 09:56:00,394 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/ed/acc585781681daa7e27505c7fa89e9040b1729/content
                2018-07-03 09:56:09,282 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/b9/bfae7246f9b55e9865e69ac6f68ee236bda02b/content
                2018-07-03 09:56:17,059 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/0a/8ba884c119675589ba4992c32fd4f67f956476/content
                2018-07-03 09:56:22,479 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/a4/0d9e42dad54a9acdc8945714afd6543dce3c69/content
                2018-07-03 09:56:26,656 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/56/e082c7c40e99db039842c54e3d31b56de0ec15/content
                2018-07-03 09:56:31,638 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/50/21084a3f617bf21b6c91eea4e153ff80a5bdd6/content
                2018-07-03 09:56:36,577 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/08/7aec1a27817e994b5b7b6bcbe2c7e4bb0ef97f/content
                2018-07-03 09:56:42,226 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/62/718700304d5d49a0263a2429c68276982ca7e8/content
                2018-07-03 09:56:44,167 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/98/870fe49b8ad4d5d50571a00b7d415a72272ec9/content
                2018-07-03 09:56:45,754 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/70/f4cdd1297ba6b4fd007ffe860eed1877d44641/content
                2018-07-03 09:56:49,212 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/10/6c3d17ed303d8df3008061fd7bc2b6cdc336f5/content
                2018-07-03 09:56:56,614 INFO  [org.jboss.as.repository] (ServerService Thread Pool -- 15) WFLYDR0001: Content added at location /opt/jboss/wildfly/standalone/data/content/b1/a2e25b1c22e1ac6b411e3f58b807f7c4fe3c0c/content
                2018-07-03 09:56:58,904 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http)
                2018-07-03 09:56:59,457 INFO  [org.xnio] (MSC service thread 1-4) XNIO version 3.6.2.Final
                2018-07-03 09:56:59,639 INFO  [org.xnio.nio] (MSC service thread 1-4) XNIO NIO Implementation Version 3.6.2.Final
                2018-07-03 09:57:00,548 INFO  [org.jboss.remoting] (MSC service thread 1-6) JBoss Remoting version 5.0.5.Final
                2018-07-03 09:57:03,074 WARN  [org.jboss.as.logging] (Controller Boot Thread) WFLYLOG0012: Replacing handler 'FILE' during add operation. Either the handler type or the module name differs from the initial configuration.
                
                
                2018-07-03 09:57:12,924    WARN  [org.jboss.as.domain.management.security] (MSC service thread 1-6)  WFLYDM0111: Keystore /opt/jboss/wildfly/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost
                2018-07-03 09:57:12,945    ERROR [stderr] (MSC service thread 1-1)  SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

                 

                At the moment, the solution we've put in place is to change the root log level from WARN to INFO.  Since doing this, the problem has not reoccurred.  Although I have some concerns over this, as I do not understand why it resolves the issue and that the issue may return at some point.

                 

                Unfortunately, I've not been able to try with Wildfly 13 as of yet.

                 

                Cheers,

                 

                Mark

                • 5. Re: Wildfly 12 datasource JNDI lookup returning null
                  cviniciusm

                  Hello,

                   

                  I think yours errors are related to:

                  "

                  2018-07-03 09:05:47,795    ERROR [stderr] (MSC service thread 1-8)  SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

                  ...

                  2018-07-03 09:57:12,945    ERROR [stderr] (MSC service thread 1-1)  SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

                   

                  Caused by: java.lang.IllegalStateException

                  at org.jboss.msc.value.InjectedValue.getValue(InjectedValue.java:47)

                  at org.jboss.as.naming.service.BinderService.getValue(BinderService.java:148)

                  at org.jboss.as.naming.service.BinderService.getValue(BinderService.java:46)

                  "

                   

                  Did you enabled sql logging to console ? If so, try to disable it to verify whether the errors are due to SLF4J misconfiguration.