4 Replies Latest reply on Apr 27, 2012 5:28 PM by jbertram

    GB of logs from hornetq on JBoss AS 7.1.1

    safetytrick

      My logs are full of messages from HornetQ like this:

       

      16:57:51,782 WARN  [org.hornetq.core.postoffice.impl.BindingsImpl] (Old I/O server worker (parentId: 1598021230, [id: 0x5f3fde6e, app04.cl01.ut.us.test-server-me.com/10.42.100.34:5445])) Couldn't find binding with id=402822 on routeFromCluster for message=ServerMessage[messageID=10737680087,priority=4, bodySize=44808,expiration=0, durable=true, address=jms.topic.topic/toplinkSynchronization,properties=TypedProperties[{}]]@2037519107 binding = BindingsImpl [name=jms.topic.topic/toplinkSynchronization]

       

      I am testing with a 6 node cluster and only one of the nodes is showing this error. What causes this warning message? I would just ignore them if there weren't so many?

       

      My messaging configuration looks like this:

       

      <subsystem xmlns="urn:jboss:domain:messaging:1.1">
          <hornetq-server>
              <clustered>true</clustered>
              <persistence-enabled>true</persistence-enabled>
              <security-enabled>false</security-enabled>
              <jmx-management-enabled>true</jmx-management-enabled>
              <message-counter-enabled>true</message-counter-enabled>
              <shared-store>false</shared-store>
              <journal-type>asyncio</journal-type>
              <journal-file-size>10485760</journal-file-size>
              <journal-min-files>2</journal-min-files>
      
              <connectors>
                  <netty-connector name="netty" socket-binding="messaging"/>
                  <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                      <param key="batch-delay" value="50"/>
                  </netty-connector>
                  <in-vm-connector name="in-vm" server-id="0"/>
              </connectors>
      
              <acceptors>
                  <netty-acceptor name="netty" socket-binding="messaging"/>
                  <netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
                      <param key="batch-delay" value="50"/>
                      <param key="direct-deliver" value="false"/>
                  </netty-acceptor>
                  <in-vm-acceptor name="in-vm" server-id="0"/>
              </acceptors>
      
              <broadcast-groups>
                  <broadcast-group name="default-broadcast-group">
                      <group-address>231.7.7.7</group-address>
                      <group-port>9876</group-port>
                      <broadcast-period>5000</broadcast-period>
                      <connector-ref>
                          netty
                      </connector-ref>
                  </broadcast-group>
              </broadcast-groups>
      
              <discovery-groups>
                  <discovery-group name="default-discovery-group">
                      <socket-binding>messaging-cluster</socket-binding>
                      <refresh-timeout>20000</refresh-timeout>
                  </discovery-group>
              </discovery-groups>
      
              <cluster-connections>
                  <cluster-connection name="default-cluster-connection">
                      <address>jms</address>
                      <connector-ref>netty</connector-ref>
                      <retry-interval>500</retry-interval>
                      <forward-when-no-consumers>true</forward-when-no-consumers>
                      <max-hops>1</max-hops>
                      <discovery-group-ref discovery-group-name="default-discovery-group"/>
                  </cluster-connection>
              </cluster-connections>
      
              <security-settings>
                  <security-setting match="#">
                      <permission type="send" roles="guest"/>
                      <permission type="consume" roles="guest"/>
                      <permission type="createnondurablequeue" roles="guest"/>
                      <permission type="deletenondurablequeue" roles="guest"/>
                  </security-setting>
              </security-settings>
      
              <address-settings>
                  <address-setting match="#">
                      <dead-letter-address>jms.queue.dlq</dead-letter-address>
                      <expiry-address>jms.queue.expiryqueue</expiry-address>
                      <redelivery-delay>1000</redelivery-delay>
                      <max-size-bytes>104857600</max-size-bytes>
                      <page-size-bytes>10485760</page-size-bytes>
                      <address-full-policy>page</address-full-policy>
                      <message-counter-history-day-limit>10</message-counter-history-day-limit>
                  </address-setting>
              </address-settings>
      
              <jms-connection-factories>
                  <connection-factory name="invmconnectionfactory">
                      <connectors>
                          <connector-ref connector-name="in-vm"/>
                      </connectors>
                      <entries>
                          <entry name="java:/connectionfactory"/>
                      </entries>
                  </connection-factory>
                  <connection-factory name="remoteconnectionfactory">
                      <connectors>
                          <connector-ref connector-name="netty"/>
                      </connectors>
                      <entries>
                          <entry name="remoteconnectionfactory"/>
                          <entry name="java:jboss/exported/jms/remoteconnectionfactory"/>
                      </entries>
                  </connection-factory>
                  <pooled-connection-factory name="hornetq-ra">
                      <transaction mode="xa"/>
                      <connectors>
                          <connector-ref connector-name="in-vm"/>
                      </connectors>
                      <entries>
                          <entry name="java:/jmsxa"/>
                      </entries>
                  </pooled-connection-factory>
              </jms-connection-factories>
      
              <jms-destinations>
                  <jms-queue name="testqueue">
                      <entry name="queue/test"/>
                      <entry name="java:jboss/exported/jms/queue/test"/>
                      <durable>false</durable>
                  </jms-queue>
                  <jms-queue name="queue/backgroundjobqueue">
                      <entry name="java:jboss/exported/jms/queue/backgroundjobqueue"/>
                      <durable>true</durable>
                  </jms-queue>
                  <jms-queue name="queue/applicationevent">
                      <entry name="java:jboss/exported/jms/queue/applicationevent"/>
                      <durable>true</durable>
                  </jms-queue>
                  <jms-topic name="testtopic">
                      <entry name="topic/test"/>
                      <entry name="java:jboss/exported/jms/topic/test"/>
                  </jms-topic>
                  <jms-topic name="topic/cachesynchronization">
                      <entry name="java:jboss/exported/jms/topic/cachesynchronization"/>
                  </jms-topic>
                  <jms-topic name="topic/toplinksynchronization">
                      <entry name="java:jboss/exported/jms/topic/toplinksynchronization"/>
                  </jms-topic>
              </jms-destinations>
          </hornetq-server>
      </subsystem>
      
      
        • 1. Re: GB of logs from hornetq on JBoss AS 7.1.1
          jbertram

          I noticed that you have <clustered>true</clustered>.  Do you have other instances of JBoss AS7 which are clustered from a HornetQ perspective?  If so, do they have all the same destinations defined as this instance?

          • 2. Re: GB of logs from hornetq on JBoss AS 7.1.1
            safetytrick

            All 6 nodes have the exact same standalone.xml and messaging section that you see above. The only difference between any of the 6 nodes is the IP they bind to.

             

            I also noticed this message on startup:

             

            16:44:23,719 WARN  [org.hornetq.core.postoffice.impl.PostOfficeImpl] (Old I/O server worker (parentId: 500849146, [id: 0x1dda59fa, app04.cl01.ut.us.attask.com/10.42.100.34:5445])) Duplicate message detected through the bridge - message will not be routed. Message information:

            ServerMessage[messageID=8590194499,priority=4, bodySize=1468,expiration=0, durable=true, address=jms.topic.topic/toplinkSynchronization,properties=TypedProperties[{_HQ_BRIDGE_DUP=[B@50691ccf, _HQ_ROUTE_TO=[B@24e28347}]]@320388412

             

            The messageID and parentId fields on all of the log messages are the same.

             

            Could there be a message stuck in a loop somewhere?

             

            Michael

            • 3. Re: GB of logs from hornetq on JBoss AS 7.1.1
              safetytrick

              Hmm, I cleared out the data directory and the warning message went away. I'd hate to get this bug in a live environment. Has you seen this issue or something similar before?

              • 4. Re: GB of logs from hornetq on JBoss AS 7.1.1
                jbertram

                I haven't see this or anything similar before.  However, if you can work up a reproducible test-case feel free to open a JIRA.