11 Replies Latest reply on Jun 19, 2013 11:40 AM by Andy Taylor

    Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone

    Florin Slev Newbie

      Conditions:

       

      Two Jboss AS 7.2.0 servers with HQ 2.3.0 CR1 inside cluster configuration.

       

      Setup a live-backup configuration, using replication:

       

      (Live-Backup config - CLI commands)

       

      Live :

       

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=shared-store, value=false)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=allow-failback, value=true)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=failover-on-shutdown, value=true)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=check-for-live-server, value=true)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=backup-group-name, value=bk_gr_name)

       

      Backup:

       

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=backup, value=true)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=shared-store, value=false)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=allow-failback, value=true)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=check-for-live-server, value=true)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=failover-on-shutdown, value=true)

      /subsystem=messaging/hornetq-server=default/:write-attribute(name=backup-group-name, value=bk_gr_name)

       

      Play a little bit with this configuration: stop  LIVE, BACKUP becomes active. Things go as expected.

       

      Undo the live backup configuration:

       

      LIVE:

       

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=shared-store)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=allow-failback)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=failover-on-shutdown)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=check-for-live-server)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=backup-group-name)

       

      BACKUP:

       

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=backup)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=shared-store)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=allow-failback)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=check-for-live-server)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=failover-on-shutdown)

      /subsystem=messaging/hornetq-server=default/:undefine-attribute(name=backup-group-name)

       

       

      Restart both servers. After restart, both servers continuously log the same warning:

       

      WARN  [org.hornetq.core.client] (hornetq-discovery-group-thread-dg-group1) HQ212050: There are more than one servers on the network broadcasting the same node id. You will see this message exactly once (per node) if a node is restarted, in which case it can be safely ignored. But if it is logged continuously it means you really do have more than one node on the same network active concurrently with the same node id. This could occur if you have a backup node active at the same time as its live node. nodeID=79d6e606-d831-11e2-9ce6-d9222ff73a66

       

      Only after removing "data" folder from the live server and restart it , the warning doesn't appear anymore.

       

      Any ideas ? Maybe I am doing something wrong ?

        • 2. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
          Florin Slev Newbie

          The servers are implicitly using different journals because they are on different physical machines. They do not share the same journal data folder.

          • 3. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
            Andy Taylor Master

            when you start both servers the backup should come up as a backup, if you are seeing this message then both servers are live, could you post your logs please?

            • 4. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
              Florin Slev Newbie

              I've attached the server logs from live and backup , when both servers are removed from the live-backup configuration, and restarted ... the las warnings are continuously logged. I am wondering if someone managed to reproduce this scenario or maybe the problem is on my side.

              • 5. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
                Andy Taylor Master

                The backup server is being started as a live server not a backup, see:

                 

                2013-06-19 15:02:24,886 INFO  [org.hornetq.core.server] (MSC service thread 1-2) HQ221001: live server is starting with configuration HornetQ Configuration (clustered=true,backup=false,sharedStore=true,journalDirectory=/opt/ui/etc/slot1/standalone/data/messagingjournal,bindingsDirectory=/opt/ui/etc/slot1/standalone/data/messagingbindings,largeMessagesDirectory=/opt/ui/etc/slot1/standalone/data/messaginglargemessages,pagingDirectory=/opt/ui/etc/slot1/standalone/data/messagingpaging)

                 

                can you check your config

                • 6. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
                  Florin Slev Newbie

                  I can see now, however, if I am running the CLI command for viewing the HQ config, I cannot see where the problem is. I believe everything should look as before the backup was configured.

                  The fact is that I am undoing the live-backup configuration, so of course the "backup" server should not behave as a backup anymore. The two servers should just be in cluster mode as before, and nothing more. The only difference is that they are  continuously logging warnings and only if I am removinf the "data" directory from jboss AS , the warning is not logged anymore.

                   

                  standalone@vm-fslevoaca-4933.development.lan:10099 /] /subsystem=messaging/hornetq-server=default/:read-resource

                  {

                      "outcome" => "success",

                      "result" => {

                          "acceptor" => undefined,

                          "allow-failback" => true,

                          "async-connection-execution-enabled" => true,

                          "backup" => false,

                          "backup-group-name" => undefined,

                          "bridge" => undefined,

                          "check-for-live-server" => false,

                          "cluster-password" => expression "${jboss.default.multicast.address:230.0.0.4}",

                          "cluster-user" => "hornetQClusterUser",

                          "clustered" => false,

                          "connection-ttl-override" => -1L,

                          "connector" => undefined,

                          "connector-service" => undefined,

                          "create-bindings-dir" => true,

                          "create-journal-dir" => true,

                          "divert" => undefined,

                          "failback-delay" => 5000L,

                          "failover-on-shutdown" => false,

                          "grouping-handler" => undefined,

                          "id-cache-size" => 15000,

                          "jms-topic" => undefined,

                          "jmx-domain" => "org.hornetq",

                          "jmx-management-enabled" => true,

                          "journal-buffer-size" => undefined,

                          "journal-buffer-timeout" => undefined,

                          "journal-compact-min-files" => 10,

                          "journal-compact-percentage" => 30,

                          "journal-file-size" => 102400L,

                          "journal-max-io" => undefined,

                          "journal-min-files" => 2,

                          "journal-sync-non-transactional" => true,

                          "journal-sync-transactional" => true,

                          "journal-type" => "ASYNCIO",

                          "log-journal-write-rate" => false,

                          "management-address" => "jms.queue.hornetq.management",

                          "management-notification-address" => "hornetq.notifications",

                          "memory-measure-interval" => -1L,

                          "memory-warning-threshold" => 25,

                          "message-counter-enabled" => false,

                          "message-counter-max-day-history" => 10,

                          "message-counter-sample-period" => 10000L,

                          "message-expiry-scan-period" => 30000L,

                          "message-expiry-thread-priority" => 3,

                          "page-max-concurrent-io" => 5,

                          "perf-blast-pages" => -1,

                          "persist-delivery-count-before-delivery" => false,

                          "persist-id-cache" => true,

                          "persistence-enabled" => true,

                          "queue" => undefined,

                          "remoting-incoming-interceptors" => undefined,

                          "remoting-interceptors" => undefined,

                          "remoting-outgoing-interceptors" => undefined,

                          "replication-clustername" => undefined,

                          "run-sync-speed-test" => false,

                          "scheduled-thread-pool-max-size" => 5,

                          "security-domain" => "other",

                          "security-enabled" => true,

                          "security-invalidation-interval" => 10000L,

                          "server-dump-interval" => -1L,

                          "shared-store" => true,

                          "thread-pool-max-size" => 30,

                          "transaction-timeout" => 300000L,

                          "transaction-timeout-scan-period" => 1000L,

                          "wild-card-routing-enabled" => true,

                          "address-setting" => {"#" => undefined},

                          "broadcast-group" => {"bg-group1" => undefined},

                          "cluster-connection" => {"my-cluster" => undefined},

                          "connection-factory" => {

                              "InVmConnectionFactory" => undefined,

                              "InVmXAConnectionFactory" => undefined,

                              "NettyConnectionFactory" => undefined,

                              "NettyXAConnectionFactory" => undefined

                          },

                          "core-address" => {

                              "jms.queue.DLQ" => undefined,

                              "jms.queue.testQueue1" => undefined,

                              "sf.my-cluster.8206ce64-d8cf-11e2-9d03-67134ae63711" => undefined,

                              "jms.queue.resultQueue" => undefined,

                              "jms.queue.liveBackupQ" => undefined,

                              "jms.queue.PsiCallbackQueue" => undefined,

                              "jms.queue.ExpiryQueue" => undefined

                          },

                          "discovery-group" => {"dg-group1" => undefined},

                          "in-vm-acceptor" => {"in-vm" => undefined},

                          "in-vm-connector" => {"in-vm" => undefined},

                          "jms-queue" => {

                              "DLQ" => undefined,

                              "testQueue1" => undefined,

                              "ExpiryQueue" => undefined

                          },

                          "path" => {

                              "journal-directory" => undefined,

                              "paging-directory" => undefined,

                              "bindings-directory" => undefined,

                              "large-messages-directory" => undefined

                          },

                          "pooled-connection-factory" => {"hornetq-ra" => undefined},

                          "remote-acceptor" => {

                              "netty" => undefined,

                              "netty-throughput" => undefined

                          },

                          "remote-connector" => {

                              "netty" => undefined,

                              "netty-throughput" => undefined

                          },

                          "runtime-queue" => {

                              "jms.queue.testQueue1" => undefined,

                              "sf.my-cluster.8206ce64-d8cf-11e2-9d03-67134ae63711" => undefined,

                              "jms.queue.DLQ" => undefined,

                              "jms.queue.liveBackupQ" => undefined,

                              "jms.queue.resultQueue" => undefined,

                              "jms.queue.ExpiryQueue" => undefined,

                              "jms.queue.PsiCallbackQueue" => undefined

                          },

                          "security-setting" => {"#" => undefined}

                      }

                  • 8. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
                    Florin Slev Newbie

                    Yes, but that's the idea. I don't want to keep the backup anymore. I want to remove the live-backup configuration and to use the two servers as normal clustered servers as before.

                    • 9. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
                      Andy Taylor Master

                      once a backup server is a backup its always a backup (until it fails over), thats how it is configured. since it is replicating the other live server it has the same journal, which is used when you delete the backup and re create it as live. you will need to restart the whole server completely after deleting the data directory.

                       

                      If you want the journal deleted when you delete the server then you will have to raise this with the AS7 guys

                      • 11. Re: Both severs from an ex live-backup configuration continuously log  warnings after the live-backup config is undone
                        Andy Taylor Master

                        yes, this is the expected behaviour, but it doesnt mean the AS7 guys cant add some code to delete the journal, raise a jira if you wish