6 Replies Latest reply on Oct 10, 2016 9:06 PM by wcomnisky

    Hawkular does not start due to ActiveMQNativeIOError: NATIVE_ERROR_CANT_INITIALIZE_AIO (Ubuntu 14.04)

    Peter Becker Newbie

      Hello,

       

      I'm trying to start Hawkular on my machine which runs Ubuntu 14.04.3. I tried the ZIP version of hawkular-1.0.0.Alpha9, as well as building it myself from git. Both result in the same issue with ActiveMQ failing to start up:

       

      13:16:20,939 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=false,journalDirectory=/home/pbecker/projects/hawkular/dist/target/hawkular-1.0.0.Alpha10-SNAPSHOT/standalone/data/activemq/journal,bindingsDirectory=/home/pbecker/projects/hawkular/dist/target/hawkular-1.0.0.Alpha10-SNAPSHOT/standalone/data/activemq/bindings,largeMessagesDirectory=/home/pbecker/projects/hawkular/dist/target/hawkular-1.0.0.Alpha10-SNAPSHOT/standalone/data/activemq/largemessages,pagingDirectory=/home/pbecker/projects/hawkular/dist/target/hawkular-1.0.0.Alpha10-SNAPSHOT/standalone/data/activemq/paging)
      13:16:20,941 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-7) WFLYDS0013: Started FileSystemDeploymentService for directory /home/pbecker/projects/hawkular/dist/target/hawkular-1.0.0.Alpha10-SNAPSHOT/standalone/deployments
      13:16:20,942 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0027: Starting deployment of "hawkular-inventory-dist.war" (runtime-name: "hawkular-inventory-dist.war")
      13:16:20,943 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0027: Starting deployment of "hawkular-alerts-actions-email.war" (runtime-name: "hawkular-alerts-actions-email.war")
      13:16:21,032 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221012: Using AIO Journal
      13:16:21,092 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-8) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.0.Final
      13:16:21,092 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-3) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.0.Final
      13:16:21,098 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-1) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.0.Final
      13:16:21,092 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-2) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.0.Final
      13:16:21,092 INFO  [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-5) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.0.Final
      13:16:21,534 INFO  [org.jboss.ws.common.management] (MSC service thread 1-6) JBWS022052: Starting JBossWS 5.1.2.Final (Apache CXF 3.1.4) 
      13:16:21,778 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE
      13:16:21,850 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP
      13:16:22,194 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ
      13:16:22,196 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP
      13:16:22,691 WARN  [org.apache.activemq.artemis.journal] (ServerService Thread Pool -- 68) AMQ142018: Temporary files were left unnatended after a crash on journal directory, deleting invalid files now
      13:16:22,692 WARN  [org.apache.activemq.artemis.journal] (ServerService Thread Pool -- 68) AMQ142019: Deleting orphaned file activemq-data-1.amq.tmp
      13:16:22,693 WARN  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ222010: Critical IO Error, shutting down the server. file=AIOSequentialFile:/home/pbecker/projects/hawkular/dist/target/hawkular-1.0.0.Alpha10-SNAPSHOT/standalone/data/activemq/journal/activemq-data-1.amq.tmp, message=Cannot open file:Invalid argument: java.io.IOException: Cannot open file:Invalid argument
        at org.apache.activemq.artemis.jlibaio.LibaioContext.open(Native Method)
        at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:290)
        at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:273)
        at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.open(AIOSequentialFile.java:133)
        at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile0(JournalFilesRepository.java:577)
        at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile(JournalFilesRepository.java:530)
        at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.ensureMinFiles(JournalFilesRepository.java:205)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.setUpCurrentFile(JournalImpl.java:2652)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1728)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1126)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1110)
        at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.loadMessageJournal(JournalStorageManager.java:1246)
        at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:1701)
        at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:1595)
        at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:60)
        at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:393)
        at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:381)
        at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:194)
        at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:63)
        at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:97)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at org.jboss.threads.JBossThread.run(JBossThread.java:320)
      
      
      13:16:22,717 ERROR [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ224000: Failure in initialisation: ActiveMQNativeIOError[errorType=NATIVE_ERROR_CANT_INITIALIZE_AIO message=Cannot open file:Invalid argument]
        at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.open(AIOSequentialFile.java:137)
        at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile0(JournalFilesRepository.java:577)
        at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.createFile(JournalFilesRepository.java:530)
        at org.apache.activemq.artemis.core.journal.impl.JournalFilesRepository.ensureMinFiles(JournalFilesRepository.java:205)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.setUpCurrentFile(JournalImpl.java:2652)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1728)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1126)
        at org.apache.activemq.artemis.core.journal.impl.JournalImpl.load(JournalImpl.java:1110)
        at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.loadMessageJournal(JournalStorageManager.java:1246)
        at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.loadJournals(ActiveMQServerImpl.java:1701)
        at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart2(ActiveMQServerImpl.java:1595)
        at org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:60)
        at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:393)
        at org.apache.activemq.artemis.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:381)
        at org.wildfly.extension.messaging.activemq.jms.JMSService.doStart(JMSService.java:194)
        at org.wildfly.extension.messaging.activemq.jms.JMSService.access$000(JMSService.java:63)
        at org.wildfly.extension.messaging.activemq.jms.JMSService$1.run(JMSService.java:97)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at org.jboss.threads.JBossThread.run(JBossThread.java:320)
      Caused by: java.io.IOException: Cannot open file:Invalid argument
        at org.apache.activemq.artemis.jlibaio.LibaioContext.open(Native Method)
        at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:290)
        at org.apache.activemq.artemis.jlibaio.LibaioContext.openFile(LibaioContext.java:273)
        at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.open(AIOSequentialFile.java:133)
        ... 22 more
      
      
      13:16:22,722 INFO  [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221001: Apache ActiveMQ Artemis Message Broker version 1.1.0.wildfly-010 [nodeID=6a4e735e-c30f-11e5-a7ac-893713ec38aa] 
      13:16:22,830 INFO  [org.apache.activemq.artemis.core.server] (pool-4-thread-1) AMQ221002: Apache ActiveMQ Artemis Message Broker version 1.1.0.wildfly-010 [6a4e735e-c30f-11e5-a7ac-893713ec38aa] stopped
      

       

      I've tried to fix this by adding {{HAWKULAR_BASE_DIR}}/modules/system/layers/base/org/apache/activemq/artemis/main/lib/linux-x86_64 to $LD_LIBRARY_PATH as well as to -Djava.library.path, but neither seems to change anything.

       

      Any advice on how to tackle this one? At the moment I'm only evaluating - is there a way to disable AIO?

       

      Cheers,

        Peter