Wildfly - Activemq Client Connection Loop

[skull]

I am using wildfly domain with profile full-ha.xml

I started 2 nodes of wildfly, then after some time I shutdown one of theses nodes.

The alive node begin give me this messages for ever every 3s

If i restart node 1 the messages stop.

How can I configure activemq to see that node1 die and stop try to reconect for ever?

I have try to change Reconnect Attempts, but no success.

[Server:server-test2] 18:52:00,021 ERROR [org.apache.activemq.artemis.core.client] (Thread-272 (ActiveMQ-client-global-threads-1173203024)) AMQ214016: Failed to create netty connection: java.net.NoRouteToHostException: No route to host: /xxx.xxx.xxx.xxx:8080
[Server:server-test2]  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[Server:server-test2]  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[Server:server-test2]  at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
[Server:server-test2]  at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[Server:server-test2]  at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
[Server:server-test2]  at java.lang.Thread.run(Thread.java:745)
[Server:server-test2]
[Server:server-test2] 18:52:03,022 ERROR [org.apache.activemq.artemis.core.client] (Thread-272 (ActiveMQ-client-global-threads-1173203024)) AMQ214016: Failed to create netty connection: java.net.NoRouteToHostException: No route to host: /xxx.xxx.xxx.xxx:8080
[Server:server-test2]  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
[Server:server-test2]  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
[Server:server-test2]  at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
[Server:server-test2]  at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[Server:server-test2]  at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[Server:server-test2]  at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
[Server:server-test2]  at java.lang.Thread.run(Thread.java:745)
[Server:server-test2]

[jbertram]

Can you post your <cluster-connection> configuration?  Also, where did you try to change reconnect-attempts?

[skull]

its the default from full-ha.xml, just try changing the reconnect-attempts

          <subsystem xmlns="urn:jboss:domain:messaging-activemq:1.0">
                <server name="default">
                    <cluster password="${jboss.messaging.cluster.password:MYPASSWORDCRIPT}"/>
                    <security-setting name="#">
                        <role name="guest" delete-non-durable-queue="true" create-non-durable-queue="true" consume="true" send="true"/>
                    </security-setting>
                    <address-setting name="#" redistribution-delay="1000" message-counter-history-day-limit="10" page-size-bytes="2097152" max-size-bytes="10485760" expiry-address="jms.queue.ExpiryQueue" dead-letter-address="jms.queue.DLQ"/>
                    <http-connector name="http-connector" endpoint="http-acceptor" socket-binding="http"/>
                    <http-connector name="http-connector-throughput" endpoint="http-acceptor-throughput" socket-binding="http">
                        <param name="batch-delay" value="50"/>
                    </http-connector>
                    <in-vm-connector name="in-vm" server-id="0"/>
                    <http-acceptor name="http-acceptor" http-listener="default"/>
                    <http-acceptor name="http-acceptor-throughput" http-listener="default">
                        <param name="batch-delay" value="50"/>
                        <param name="direct-deliver" value="false"/>
                    </http-acceptor>
                    <in-vm-acceptor name="in-vm" server-id="0"/>
                    <broadcast-group name="bg-group1" connectors="http-connector" jgroups-channel="activemq-cluster"/>
                    <discovery-group name="dg-group1" jgroups-channel="activemq-cluster"/>
                    <cluster-connection name="my-cluster" discovery-group="dg-group1" reconnect-attempts="10" connector-name="http-connector" address="jms"/>
                    <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/>
                    <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/>
                    <connection-factory name="InVmConnectionFactory" reconnect-attempts="10" entries="java:/ConnectionFactory" connectors="in-vm"/>
                    <connection-factory name="RemoteConnectionFactory" reconnect-attempts="-1" block-on-acknowledge="true" ha="true" entries="java:jboss/exported/jms/RemoteConnectionFactory" connectors="http-connector"/>
                    <pooled-connection-factory name="activemq-ra" transaction="xa" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory" connectors="in-vm"/>
                </server>
            </subsystem>

[jbertram]

Do you have any clients running on either of the cluster nodes?

[skull]

i have an app with annotation

@JMSDestinationDefinition(name = JMSUserControllSender.jsmTopic, interfaceName = "javax.jms.Topic", destinationName = JMSUserControllSender.jsmTopicDestination)

@JMSDestinationDefinition(name = SystemUtil.jsmTopic, interfaceName = "javax.jms.Topic", destinationName = SystemUtil.jsmTopicDestination)

this count like a client?

[jbertram]

If the app is using the destinations defined by those annotations then yes.  My next question is how are the client's connections configured?

[skull]

is this what you want to know?

package com.test.app.core.jms;


import java.util.Calendar;


import javax.ejb.Stateless;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.JMSContext;
import javax.jms.JMSDestinationDefinition;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageListener;
import javax.jms.Topic;
import javax.jms.TopicConnectionFactory;
import javax.jms.TopicSession;
import javax.jms.TopicSubscriber;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.servlet.http.HttpSession;


import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;


import com.test.app.core.security.UserPrincipal;
import com.test.app.core.security.UserPrincipalImpl;
import com.test.app.core.security.session.UserControlCommand;


@JMSDestinationDefinition(name = JMSUserControllSender.jsmTopic, interfaceName = "javax.jms.Topic", destinationName = JMSUserControllSender.jsmTopicDestination)
@Stateless
public class JMSUserControllSender implements MessageListener {
  /**
  * Cria os objetos para trabalhar com o JMS.<br>
  * Sincronismo do websocket em cluster
  */
  private ConnectionFactory connectionFactory;
  private Topic topic;
  private Connection conn;
  protected static final String jsmTopic = "java:/jms/topic/wizfee/usercontrol";
  protected static final String jsmTopicDestination = "userControllTopic";

  private static final Log log = LogFactory.getLog(JMSUserControllSender.class);

  private static JMSUserControllSender instance = new JMSUserControllSender();

  private JMSUserControllSender() {
  }

  public static JMSUserControllSender getInstance() {
  return JMSUserControllSender.instance;
  }

  /**
  * Inicializa o listener do JMS
  */
  public void start() {

  try {
  // TODO: rever o Connection para TOMEE
  this.connectionFactory = InitialContext.doLookup("java:/ConnectionFactory");
  this.topic = InitialContext.doLookup(JMSUserControllSender.jsmTopic);

  TopicConnectionFactory topicConnectionFactory = (TopicConnectionFactory) connectionFactory;
  this.conn = topicConnectionFactory.createConnection();
  TopicSession session = (TopicSession) conn.createSession(false, TopicSession.AUTO_ACKNOWLEDGE);
  TopicSubscriber consumer = session.createSubscriber(topic);
  consumer.setMessageListener(this);
  this.conn.start();
  } catch (JMSException | NamingException e) {
  JMSUserControllSender.log.error("Could not load JMS: " + e.getMessage());
  }
  }

  private void send(UserControllSession message) {
  if (this.connectionFactory != null && this.topic != null) {
  JMSContext context = this.connectionFactory.createContext();

  context.createProducer().send(topic, message);
  context.close();
  }
  }

  @Override
  public void onMessage(Message message) {
  UserControllSession userControl;
  try {
  if (JMSUserControllSender.log.isDebugEnabled()) {
  Calendar c2 = Calendar.getInstance();
  c2.setTimeInMillis(message.getJMSTimestamp());
  JMSUserControllSender.log.debug("USER CONTROLL RECEIVED: " + c2.getTime());
  }
  userControl = message.getBody(UserControllSession.class);
  UserControlCommand u = new UserControlCommand(userControl.getSessionId(), userControl.getUserName(), userControl.getPrincipal());
  u.execute(null,true);
  JMSUserControllSender.log.debug("USER CONTROLL FINISH");
  } catch (Exception e1) {
  e1.printStackTrace();
  }
  }

  public void dispatcher(HttpSession session, String username, UserPrincipalImpl UserPrincipal) {
  this.dispatcher(session, username, UserPrincipal, false);
  }

  public void dispatcher(HttpSession session, String username, UserPrincipal principal, boolean remove) {
  String sessionId = session.getId();
  if (remove) {
  sessionId = null;
  }
  this.send(new UserControllSession(sessionId, username, principal));
  }

  public void stop() {
  try {
  this.conn.stop();
  } catch (JMSException e) {
  e.printStackTrace();
  }
  try {
  this.conn.close();
  } catch (JMSException e) {
  e.printStackTrace();
  }
  }

}

[mnovak]

Could you try configure the topics in full-ha profile? Annotation JMSDestinationDefinition might be tricky as before shutdown it also undeploys the EJB with all destinations.

It can be configured messaging-activemq subsystem like:

<jms-topic name="InTopic" entries="jms/topic/InTopic java:jboss/exported/jms/topic/InTopic"/>