3 Replies Latest reply on Jun 5, 2015 8:11 PM by jbertram

    Clustering - shared store

    loganheights

      I have two hornetQ servers set up to access the same cifs mount. However, the files that are created by one are not accessible by the other. For example, if I start the live one first, it creates the journal/binding/etc directories on the shared drive (e.g. /shared). However, when I subsequently start the backup, it will report a java io permissions error. I went round and round with this until I started the backup first, which came up fine, then the "live" one second...this time the live one reported the io error.

       

      At this point, I configured each server to point to a subdirectory on the cifs mount: /shared/ms1, /shared/ms2. Now they both come up without error but this isn't the shared store approach detailed here:

      Chapter 39. High Availability and Failover

       

      My guess is that the first server (either backup or live) to come up and create the files on /shared is locking them such that the second one can't access them.

       

      Any suggestions?

        • 1. Re: Clustering - shared store
          jbertram

          The "live" server should indeed lock the journal so that only one server is ever accessing the journal at a time.  However, the "backup" should simply wait for the lock to be released (indicating the live server has died and that it should take over).  The "backup" should not report any errors during this process.  It should just wait silently.  My guess is that your problems are either caused by the way your CIFS mount is configured or by the way the JVM is working with it.  In either case the issue is not something that can be solved by HornetQ itself. 

           

          Other questions:

          • Have you tried NFS?
          • What's the full stack-trace of the error?
          • 2. Re: Clustering - shared store
            loganheights

            Hi Justin,

             

            Have not tried NFS. All of our servers connect via cifs, not sure if NFS would even be allowed.

             

            Here is the stack trace (again, this error doesn't occur if I point the shared store to its own location on the cifs mount):

             

            16:23:51,148 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on 0.0.0.0:4447

            16:23:51,148 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) JBAS017100: Listening on 0.0.0.0:9999

            16:23:51,306 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 62) MSC000001: Failed to start service jboss.messaging.default.jms.manager: org.jboss.msc.service.StartException in service jboss.messaging.default.jms.manager: JBAS011644: Failed to start service

                    at org.jboss.as.messaging.jms.JMSService.doStart(JMSService.java:159)

                    at org.jboss.as.messaging.jms.JMSService.access$000(JMSService.java:60)

                    at org.jboss.as.messaging.jms.JMSService$1.run(JMSService.java:94)

                    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [rt.jar:1.8.0_25]

                    at java.util.concurrent.FutureTask.run(Unknown Source) [rt.jar:1.8.0_25]

                    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [rt.jar:1.8.0_25]

                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [rt.jar:1.8.0_25]

                    at java.lang.Thread.run(Unknown Source) [rt.jar:1.8.0_25]

                    at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final-redhat-1.jar:2.1.1.Final-redhat-1]

            Caused by: java.io.IOException: Permission denied

                    at sun.nio.ch.FileDispatcherImpl.pread0(Native Method) [rt.jar:1.8.0_25]

                    at sun.nio.ch.FileDispatcherImpl.pread(Unknown Source) [rt.jar:1.8.0_25]

                    at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source) [rt.jar:1.8.0_25]

                    at sun.nio.ch.IOUtil.read(Unknown Source) [rt.jar:1.8.0_25]

                    at sun.nio.ch.FileChannelImpl.readInternal(Unknown Source) [rt.jar:1.8.0_25]

                    at sun.nio.ch.FileChannelImpl.read(Unknown Source) [rt.jar:1.8.0_25]

                    at org.hornetq.core.server.NodeManager.createNodeId(NodeManager.java:253)

                    at org.hornetq.core.server.NodeManager.setUpServerLockFile(NodeManager.java:236)

                    at org.hornetq.core.server.impl.FileLockNodeManager.start(FileLockNodeManager.java:67)

                    at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:412)

                    at org.hornetq.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:486)

                    at org.jboss.as.messaging.jms.JMSService.doStart(JMSService.java:155)

                    ... 8 more

             

             

            16:23:51,544 INFO  [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report

            JBAS014777:   Services which failed to start:      service jboss.messaging.default.jms.manager: org.jboss.msc.service.StartException in service jboss.messaging.default.jms.manager: JBAS011644: Failed to start service

            • 3. Re: Clustering - shared store
              jbertram

              Based on the stack-trace I'd say this is a problem with CIFS.  If you can't try NFS then I'd recommend using replication rather than a share store.

              1 of 1 people found this helpful