-
1. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
jfclere Aug 5, 2013 7:34 AM (in response to antlia)You need to wait a few seconds (~ 10) until the node are back in the OK state. (load !=-1).
Did you wait enough?
-
2. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
antlia Aug 5, 2013 8:58 AM (in response to jfclere)Hi Jean-Frederic,
I waited for ~ 60 secs, but the message still appears when refreshing the page, the only way for restore the correct state of execution is to restart apache service in Load Balancer machine.
Edit:
While seeing 503 error, mod_cluster-manager correctly shows all the nodes in the cluster, each of them having load factor != -1
-
3. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
rhusar Aug 5, 2013 9:46 AM (in response to antlia)What version are you using?
Is there any more relevant info if you turn on the debug log in httpd? (LogLevel debug)
-
4. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
antlia Aug 5, 2013 10:42 AM (in response to rhusar)Hi Radolslav, I'm using the following versions of mod_cluster and JBoss AS:
- mod_cluster 1.2.0 Final
- JBoss AS 7.1.1 Final
I've replied the issue, after changing Apache LogLevel to debug as you suggested.
The error.log attached to this message, shows the following test:
- Startup JBoss in domain mode (single node)
- Startup Apache (wait some time for cluster configuration completion in mod_cluster)
- Load mod_cluster-manager page: everything is fine (node listed, application correctly deployed, load != -1)
- Web application page loaded succesfully
- Stop JBoss node from JBoss admin console (server-three stopped correctly)
- Refresh mod_cluster-manager view: cluster correctly refreshed, node-list empty
- Restart JBoss node from JBoss admin console: node succesfully restarted
- Refresh mod_cluster-manager view: cluster correctly refreshed, one element present in node list
- Reload web applicatoin page: 503 Error
In error.log, before "All workers in error state error", now I can see: [debug] mod_proxy_cluster.c(1543): proxy: byrequests balancer FAILED
Another interesting thing: during the first request, the log also reports this warn: [warn] manager_handler STATUS error: MEM: Can't read node, but I've configured JBoss with instance-id="${jboss.node.name" attribute, which is (as the JBoss documentation says), the default workaround for avoiding such message...so I'm not able to explain this warn...
Can you help me?
Best Regards
-
error.log.zip 2.8 KB
-
-
5. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
jfclere Aug 6, 2013 2:30 AM (in response to antlia)The [warn] manager_handler STATUS error: MEM: Can't read node are normal when you start or restart AS.
I don't find anything obvious in the error_log file.
Are you using ProxyPass in the httpd.conf?
-
6. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
antlia Aug 6, 2013 4:28 AM (in response to jfclere)Hi Jean Frederick,
i tried both with CreateBalancer 1 + ProxyPass directives, and without these instructions...but none of these soluctions do the trick...here I've attached the configuration files I'm currently using in Apache (apache2 on Ubuntu).
-
apache_cfg.zip 9.1 KB
-
-
7. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
jfclere Aug 6, 2013 5:07 AM (in response to antlia)you shouldn't mix ProxyPass and mod_cluster (at least with 1.2.0.Final). What the log with ProxyPasss or without?
-
8. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
antlia Aug 6, 2013 6:50 AM (in response to jfclere)Sorry Jean Frederick i didn't understand, should I remove all ProxyPass and ProxyPassReverse directives from my apache's config files? Can I keep ProxyPreserveHost?
Why ProxyPass directives can cause problems? Is there a bug in mod_cluster that prevents its normal behavior?
However, I made the test again, removing CreateBanlancers, ProxyPass, ProxyPassReverse and ProxyPreserveHost directives, the test is the same I made yesterday, described by the previous post, and this is apache's error.log
Thanks for the support.
-
error.log.zip 3.7 KB
-
-
9. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
jfclere Aug 6, 2013 11:12 AM (in response to antlia)should I remove all ProxyPass and ProxyPassReverse directives from my apache's config files?
Yes
Can I keep ProxyPreserveHost?
Yes
There were bugs in the old version of mod_cluster that affected the mod_proxy ProxyPass directives.
I can't find any error in the error.log. Could you compile a never version .1.2.x of mod_cluster and retry?
-
10. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
antlia Aug 6, 2013 11:58 AM (in response to jfclere)I forgot to mention the fact that I'm using x64 version of mod_cluster libraries, downloaded directly from mod_cluster official site.
I'll try to download the binaries and compile them, directly from my machine.
Thanks for your support.
-
11. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
jfclere Aug 6, 2013 12:02 PM (in response to antlia)I suggest you compile the source directly from the git repos (use the 1.2.x) branch.
-
12. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
antlia Aug 7, 2013 6:39 AM (in response to jfclere)Hi Jean-Frederic,
i tried to compile the source directly from the git repos with no luck...I also tried to install apache on a x86-based system (maybe x64 libraries are affected by some bug), nothing to do, the problem remains the same.
Maybe it can be an issue on my JBoss configuration?
I've disabled auto-announce, listing the proxies manually in domain.xml and also ajp protocol for mod_cluster messages (anyway I'm having the error even using http protocol)...is there any other option to enable/tune for mod_cluster's to work correctly?
-
13. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
jfclere Aug 7, 2013 7:43 AM (in response to antlia)"i tried to compile the source directly from the git repos with no luck.."
Does it mean that 1.2.x from the git repo has the same problem?
-
14. Re: Stopping and restarting JBoss nodes cause 503 error in mod_cluster
antlia Aug 7, 2013 8:43 AM (in response to jfclere)Yes, sorry for the poor explanation...yes, I mean that I have compiled succesfully the sources from the git repository, obtaining the same issue when starting/stopping JBoss nodes.