Using GlusterFS with HornetQ
Hi,
I am experimenting with GlusterFS to use that as the shared file system in a live/backup cluster with two nodes, with the /data directory as a glusterfs mount, accessible by both nodes.
When starting up the live node, the following output is generated :
$ ./run.sh ../config/stand-alone/non-clustered
***********************************************************************************
java -XX:+UseParallelGC -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -Xms512M -Xmx1024M -Dhornetq.config.dir=../config/stand-alone/non-clustered -Djava.util.logging.config.file=../config/stand-alone/non-clustered/logging.properties -Djava.library.path=. -classpath ../lib/twitter4j-core.jar:../lib/netty.jar:../lib/jnpserver.jar:../lib/jnp-client.jar:../lib/jboss-mc.jar:../lib/jboss-jms-api.jar:../lib/hornetq-twitter-integration.jar:../lib/hornetq-spring-integration.jar:../lib/hornetq-logging.jar:../lib/hornetq-jms.jar:../lib/hornetq-jms-client-java5.jar:../lib/hornetq-jms-client.jar:../lib/hornetq-jboss-as-integration.jar:../lib/hornetq-core.jar:../lib/hornetq-core-client-java5.jar:../lib/hornetq-core-client.jar:../lib/hornetq-bootstrap.jar:../config/stand-alone/non-clustered:../schemas/ org.hornetq.integration.bootstrap.HornetQBootstrapServer hornetq-beans.xml
***********************************************************************************
[main] 11:04:33,624 INFO [org.hornetq.integration.bootstrap.HornetQBootstrapServer] Starting HornetQ Server
[main] 11:04:35,627 INFO [org.hornetq.core.server.impl.HornetQServerImpl] live server is starting with configuration HornetQ Configuration (clustered=false,backup=false,sharedStore=true,journalDirectory=../data/journal,bindingsDirectory=../data/bindings,largeMessagesDirectory=../data/large-messages,pagingDirectory=../data/paging)
[main] 11:04:35,628 INFO [org.hornetq.core.server.impl.HornetQServerImpl] Waiting to obtain live lock
[main] 11:04:35,681 INFO [org.hornetq.core.persistence.impl.journal.JournalStorageManager] Using AIO Journal
[main] 11:04:35,713 WARNING [org.hornetq.core.server.impl.HornetQServerImpl] Security risk! It has been detected that the cluster admin user and password have not been changed from the installation default. Please see the HornetQ user guide, cluster chapter, for instructions on how to do this.
[main] 11:04:36,280 INFO [org.hornetq.core.server.impl.AIOFileLockNodeManager] Waiting to obtain live lock
[main] 11:04:36,284 INFO [org.hornetq.core.server.impl.AIOFileLockNodeManager] Live Server Obtained live lock
[main] 11:04:36,749 SEVERE [org.hornetq.core.server.impl.HornetQServerImpl] Failure in initialisation
HornetQException[errorCode=205 message=Can't open file]
at org.hornetq.core.asyncio.impl.AsynchronousFileImpl.init(Native Method)
at org.hornetq.core.asyncio.impl.AsynchronousFileImpl.open(AsynchronousFileImpl.java:207)
at org.hornetq.core.journal.impl.AIOSequentialFile.open(AIOSequentialFile.java:190)
at org.hornetq.core.journal.impl.JournalFilesRepository.createFile(JournalFilesRepository.java:581)
at org.hornetq.core.journal.impl.JournalFilesRepository.ensureMinFiles(JournalFilesRepository.java:203)
at org.hornetq.core.journal.impl.JournalImpl.load(JournalImpl.java:2165)
at org.hornetq.core.journal.impl.JournalImpl.load(JournalImpl.java:1496)
at org.hornetq.core.journal.impl.JournalImpl.load(JournalImpl.java:1480)
at org.hornetq.core.persistence.impl.journal.JournalStorageManager.loadMessageJournal(JournalStorageManager.java:841)
at org.hornetq.core.server.impl.HornetQServerImpl.loadJournals(HornetQServerImpl.java:1619)
at org.hornetq.core.server.impl.HornetQServerImpl.initialisePart2(HornetQServerImpl.java:1469)
at org.hornetq.core.server.impl.HornetQServerImpl.access$100(HornetQServerImpl.java:132)
at org.hornetq.core.server.impl.HornetQServerImpl$SharedStoreLiveActivation.run(HornetQServerImpl.java:356)
at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:570)
at org.hornetq.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:275)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)
at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)
at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241)
at org.jboss.kernel.plugins.dependency.ExecutionWrapper.execute(ExecutionWrapper.java:47)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContextAction.java:109)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:70)
at org.jboss.kernel.plugins.dependency.LifecycleAction.installActionInternal(LifecycleAction.java:221)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:774)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:540)
at org.jboss.kernel.plugins.deployment.AbstractKernelDeployer.deployBean(AbstractKernelDeployer.java:319)
at org.jboss.kernel.plugins.deployment.AbstractKernelDeployer.deployBeans(AbstractKernelDeployer.java:297)
at org.jboss.kernel.plugins.deployment.AbstractKernelDeployer.deploy(AbstractKernelDeployer.java:130)
at org.jboss.kernel.plugins.deployment.xml.BeanXMLDeployer.deploy(BeanXMLDeployer.java:96)
at org.hornetq.integration.bootstrap.HornetQBootstrapServer.deploy(HornetQBootstrapServer.java:236)
at org.hornetq.integration.bootstrap.HornetQBootstrapServer.deploy(HornetQBootstrapServer.java:206)
at org.hornetq.integration.bootstrap.HornetQBootstrapServer.bootstrap(HornetQBootstrapServer.java:155)
at org.jboss.kernel.plugins.bootstrap.AbstractBootstrap.run(AbstractBootstrap.java:83)
at org.hornetq.integration.bootstrap.HornetQBootstrapServer.run(HornetQBootstrapServer.java:116)
at org.hornetq.integration.bootstrap.HornetQBootstrapServer.main(HornetQBootstrapServer.java:73)
[main] 11:04:36,755 INFO [org.hornetq.core.server.impl.HornetQServerImpl] HornetQ Server version 2.2.5.Final (HQ_2_2_5_FINAL_AS7, 121) [71262c1d-8f2e-11e1-8263-005056a80fd0] started
The GlusterFS filesystem is mounted and working fine:
myhost.mydomain.com:gvol1 on /opt/hornetq-2.2.5.Final/data type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072)
If I unmount the GlusterFS file system and just leave the mountpoint as a directory, Hornetq comes up without any issues.
I did read that NFS mounts are not advised, but I was wondering whether someone has had experience or has advice on using GlusterFS.
Thanks,
Jan.
