JBoss 7, HornetQ, Restart backup HornetQ after failback

[mevans7]

I have two JBoss 7.2 servers (EAP 6.1).  I've built and incorporated HornetQ 2.3.12.Final into the JBoss and found it to work better with failover (than the 2.3.1.Final that comes with it).  One server is configured as "live" and the second as "backup".  They are configured for "data replication".

Scenario:

- Start live server

- Start backup server

- Shutdown live server (hornetQ fails over to backup server)

- Start live server (hornetQ fails back to live server)

HornetQ quits on backup server.

My question: How do I restart the HornetQ running in the backup server JBoss without having to restart the entire JBoss?

I suspect that CLI may be used for this function, but I cannot find out how to do that.  Is it possible?

Also, I see this bug report:  https://bugzilla.redhat.com/show_bug.cgi?id=1013536

The discussion there casts some doubt as to whether or not the HornetQ can be started within the JBoss from CLI.  That discussion also speculates about being able to configure the backup server to automatically restart the hornetQ server.

Bottom line: I need a reliable clustered JBoss/hornetQ solution.

[synclpz]

I'm also trying to achieve your goal - reliable clustered messaging. At the moment only I have achieved is reliable clustered mess

As for your question, I see that a post from one of developers could help (but I did't try myself): https://github.com/andytaylor/HornetQ-AS7-examples/tree/master/domain-replication

Here he says that we should do next thing from JBoss-CLI:

/host=master/server-config=hornetq-live:stop
  {
  "outcome" => "success",
  "result" => "STOPPING"
  }

[mevans7]

Thanks Viktor.  I saw that article.  It appears to be written for "domain" mode rather than "standalone" mode.  If I run that command, I get the following result:

[standalone@myhost:9999 /] /host=master/server-config=hornetq-live:stop

{

    "outcome" => "failed",

    "failure-description" => "JBAS014883: No resource definition is registered for address [

    (\"host\" => \"master\"),

    (\"server-config\" => \"hornetq-live\")

]",

    "rolled-back" => true

}

Good luck with your clustered messaging.

[jbertram]

HornetQ 2.3.12.Final should have all the changes necessary to allow the broker to restart automatically after fail-back (see https://github.com/hornetq/hornetq/commit/8d91a31b0f1a8cb7a8584b6472477c17a4474ae0). 

[mevans7]

Thank you Justin!  Is this change in a 2.3.x version (tag) of HornetQ that I can incorporate into my JBoss 7?  Looking at github, the most recent label on 2.3 appears to be 2.3.12.Final, and I downloaded and installed that last week.  What tag are these changes made under?

[jbertram]

As I indicated, this change is already in 2.3.12.Final which you are already apparently using.  Therefore, I'm not sure why it isn't working for you.  Have you tried configuring 2 standalone HornetQ 2.3.12.Final servers and connecting to those from your applications running in the application server?

[mevans7]

The bug report here https://bugzilla.redhat.com/show_bug.cgi?id=1013536 says:

      jbertram 2013-10-25 17:29:38 EDT

     FYI - Andy's fix added support for <max-saved-replicated-journals-size> to HornetQ,

     but from what I can see the messaging subsystem in EAP 6 was *not* updated to

     support this new parameter.  Therefore, users will be stuck with the default value for

     max-saved-replicated-journals-size (i.e. 2) until the messaging subsystem is updated

     appropriately.

So I was thinking that this value could not be passed through the JBoss 7 configuration to HornetQ.

Are you suggesting I run HornetQ outside of JBoss, and if so, are you suggesting it as a deliverable solution?  I have to think about that.  What about message driven beans?  Other issues?

(Thank you for your responses, and putting up with my careless analysis of your prior comment -- you clearly stated HQ 2.3.12.Final had the fix -- I let myself get distracted by the bugzilla report link.)

[jbertram]

So I was thinking that this value could not be passed through the JBoss 7 configuration to HornetQ.

You're right.  It can't - at least not in the version you're using.  However, that shouldn't stop the default value from being used.

Are you suggesting I run HornetQ outside of JBoss, and if so, are you suggesting it as a deliverable solution?  I have to think about that.  What about message driven beans?  Other issues?

I am suggesting that if you want this to work you should consider trying running HornetQ standalone (i.e. outside of the application server).  I'm not sure what you mean by "deliverable solution".  MDBs can still run in the application server and connect remotely to the HornetQ servers.

[bbuterbrott]

Miroslav Novak in his comment to bz (https://bugzilla.redhat.com/show_bug.cgi?id=1013536#c15) said that he added this property using the CLI. He was speaking about JBoss EAP 6.2.0? Because when I tried to add it myself in JBoss EAP 6.1.0 using this CLI command:

{
    "address" => [
        ("subsystem" => "messaging"),
        ("hornetq-server" => "test-hornetq-1-backup")
    ],
    "operation" => "write-attribute",
    "name" => "max-saved-replicated-journal-size",
    "value" => -1
}

I got this outcome, stating that this attribute is not known:

{
    "outcome" => "failed",
    "failure-description" => "JBAS014792: Unknown attribute max-saved-replicated-journal-size",
    "rolled-back" => true
}

I would like to know if there is a way to set this new property in JBoss EAP 6.1.0. Or maybe a way to tweak it default value to -1?

[jbertram]

There is no way to set this EAP 6.1 because it doesn't even exist in EAP 6.1.  If you want to use this then move to EAP 6.2.

[synclpz]

Justin, is this specific to some EAP libraries? We have already updated HornetQ jars to 2.3.12. May be some updates could solve the problem?

[jbertram]

Yes, this is specific to EAP.  All CLI or XML based administration is performed by EAP itself and then enacted upon the underlying HornetQ implementation.

[synclpz]

Got it. Is that option available in WildFly?

[jbertram]

It's not available in any release of Wildfly, but you can pull the latest snapshot build it if you want or just download a recent nightly build.

Also, as I indicated previously, it is in EAP 6.2 which is available for download as well.

[synclpz]

Thanks. We will try nighly wildfly.