1 Reply Latest reply on Feb 28, 2019 4:37 PM by stephan_b

    Wildfly(8.2.0 final) clustering not working with S3_PING on AWS

    julie333

      Hello,

       

      I tried to setup a live/backend(HA) wildfly servers on AWS using S3_PING for a couple of days and no success. And S3 bucket has two files from both live and backup server for addresses when servers are up. I am able to test S3 with awscli on both machines(seems for me the S3 bucket configured fine). But live server always got "channel is closed, unable to start broadcast group bg-group1 and unable to start cluster connection my-cluster" exceptions. Backup server looks ok. And i can't figure out why the clustering can't setup correctly. Any help is highly appreciated! I attached standalone-full-ha.xml file here for both live and backup server. And below is the log for live server which has the exception:

       

      15:59:48,078 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-7) JBAS010400: Bound data source [java:jboss/datasources/ExampleDS]

      15:59:48,313 INFO  [org.jboss.as.server.deployment.scanner] (MSC service thread 1-8) JBAS015012: Started FileSystemDeploymentService for directory /wildfly/wildfly8/standalone/deployments

      15:59:48,462 INFO  [org.jboss.ws.common.management] (MSC service thread 1-5) JBWS022052: Starting JBoss Web Services - Stack CXF Server 4.3.2.Final

      15:59:49,441 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221000: live server is starting with configuration HornetQ Configuration (clustered=true,backup=false,sharedStore=false,journalDirectory=/wildfly/wildfly8/standalone/data/journal,bindingsDirectory=/wildfly/wildfly8/standalone/data/bindings,largeMessagesDirectory=/wildfly/wildfly8/standalone/data/large-messages,pagingDirectory=/wildfly/wildfly8/standalone/data/paging)

      15:59:49,475 INFO  [stdout] (ServerService Thread Pool -- 60)

      15:59:49,475 INFO  [stdout] (ServerService Thread Pool -- 60) -------------------------------------------------------------------

      15:59:49,475 INFO  [stdout] (ServerService Thread Pool -- 60) GMS: address=ip-10-144-104-213/hornetq_broadcast_channel, cluster=hornetq_broadcast_channel, physical address=10.144.104.213:7600

      15:59:49,476 INFO  [stdout] (ServerService Thread Pool -- 60) -------------------------------------------------------------------

      16:00:59,682 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221012: Using AIO Journal

      16:00:59,713 INFO  [io.netty.util.internal.PlatformDependent] (ServerService Thread Pool -- 60) Your platform does not provide complete low-level API for accessing direct buffers reliably. Unless explicitly requested, heap buffer will always be preferred to avoid potential system unstability.

      16:00:59,746 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221043: Adding protocol support CORE

      16:00:59,750 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221043: Adding protocol support AMQP

      16:00:59,753 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221043: Adding protocol support STOMP

      16:01:01,102 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221020: Started Netty Acceptor version unknown 10.144.104.213:5445

      16:01:01,108 INFO  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ221020: Started Netty Acceptor version unknown 10.144.104.213:5455

      16:01:01,109 WARN  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ222116: unable to start broadcast group bg-group1: java.lang.IllegalStateException: channel is closed

        at org.jgroups.JChannel.checkClosed(JChannel.java:902) [jgroups-3.4.5.Final.jar:3.4.5.Final]

        at org.jgroups.JChannel._preConnect(JChannel.java:522) [jgroups-3.4.5.Final.jar:3.4.5.Final]

        at org.jgroups.JChannel.connect(JChannel.java:284) [jgroups-3.4.5.Final.jar:3.4.5.Final]

        at org.jgroups.JChannel.connect(JChannel.java:275) [jgroups-3.4.5.Final.jar:3.4.5.Final]

        at org.hornetq.api.core.JGroupsBroadcastGroupConfiguration$JGroupsBroadcastEndpoint$JChannelWrapper.connect(JGroupsBroadcastGroupConfiguration.java:331) [hornetq-core-client-2.4.5.Final.jar:]

        at org.hornetq.api.core.JGroupsBroadcastGroupConfiguration$JGroupsBroadcastEndpoint.internalOpen(JGroupsBroadcastGroupConfiguration.java:218) [hornetq-core-client-2.4.5.Final.jar:]

        at org.hornetq.api.core.JGroupsBroadcastGroupConfiguration$JGroupsBroadcastEndpoint.openBroadcaster(JGroupsBroadcastGroupConfiguration.java:186) [hornetq-core-client-2.4.5.Final.jar:]

        at org.hornetq.core.server.cluster.impl.BroadcastGroupImpl.start(BroadcastGroupImpl.java:107) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.core.server.cluster.ClusterManager.start(ClusterManager.java:244) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.core.server.impl.HornetQServerImpl.initialisePart2(HornetQServerImpl.java:1751) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.core.server.impl.HornetQServerImpl.access$1400(HornetQServerImpl.java:171) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.core.server.impl.HornetQServerImpl$SharedNothingLiveActivation.run(HornetQServerImpl.java:2725) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:451) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:488) [hornetq-jms-server-2.4.5.Final.jar:]

        at org.jboss.as.messaging.jms.JMSService.doStart(JMSService.java:169) [wildfly-messaging-8.2.0.Final.jar:8.2.0.Final]

        at org.jboss.as.messaging.jms.JMSService.access$000(JMSService.java:63) [wildfly-messaging-8.2.0.Final.jar:8.2.0.Final]

        at org.jboss.as.messaging.jms.JMSService$1.run(JMSService.java:96) [wildfly-messaging-8.2.0.Final.jar:8.2.0.Final]

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65]

        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65]

        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65]

        at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final.jar:2.1.1.Final]

       

      16:01:01,114 WARN  [org.hornetq.core.server] (ServerService Thread Pool -- 60) HQ222117: unable to start cluster connection my-cluster: HornetQInternalErrorException[errorType=INTERNAL_ERROR message=HQ119004: Failed to initialise session factory]

        at org.hornetq.core.client.impl.ServerLocatorImpl.initialise(ServerLocatorImpl.java:390) [hornetq-core-client-2.4.5.Final.jar:]

        at org.hornetq.core.client.impl.ServerLocatorImpl.start(ServerLocatorImpl.java:619) [hornetq-core-client-2.4.5.Final.jar:]

        at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl.activate(ClusterConnectionImpl.java:648) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.core.server.cluster.impl.ClusterConnectionImpl.start(ClusterConnectionImpl.java:385) [hornetq-server-2.4.5.Final.jar:]

        at org.hornetq.core.server.cluster.ClusterManager.start(ClusterManager.java:256) [hornetq-server-2.4.5.Final.jar:]

       

      Thanks a lot

      Julie

        • 1. Re: Wildfly(8.2.0 final) clustering not working with S3_PING on AWS
          stephan_b

          I just implemented a tcp/tcpping/replication/unicast cluster, and all my nodes start up with that same message:

          unable to start broadcast group bg-group1: java.lang.IllegalStateException: channel is closed

           

          My application seems to work fine, but I wish I could get rid of it. I'm seeing this in 8.2.1, which unfortunately is the latest supported version by my application.

          I know this is from 2015, but did you ever get this resolved, or is it a nuisance error?