Connection refuled after restart Database server

[jongchai]

Hi,

My application on EAP can not connect to database server after restart database service.

But my application working again after restart Jboss service.

Please help suggest me for how to investigate this issue.

13:40:46,396 WARN  [org.directwebremoting.impl.DefaultRemoter] (http-/10.1.215.206:8980-41) Method execution failed: : java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_80]

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_80]

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_80]

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_80]

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_80]

at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_80]

at sun.net.NetworkClient.doConnect(NetworkClient.java:175) [rt.jar:1.7.0_80]

at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) [rt.jar:1.7.0_80]

at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) [rt.jar:1.7.0_80]

at sun.net.www.http.HttpClient.<init>(HttpClient.java:211) [rt.jar:1.7.0_80]

at sun.net.www.http.HttpClient.New(HttpClient.java:308) [rt.jar:1.7.0_80]

at sun.net.www.http.HttpClient.New(HttpClient.java:326) [rt.jar:1.7.0_80]

at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997) [rt.jar:1.7.0_80]

at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:933) [rt.jar:1.7.0_80]

at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851) [rt.jar:1.7.0_80]

at com.kcs.ktam.dwr.HealthCheckWSManager.checkPortList(HealthCheckWSManager.java:66) [classes:]

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_80]

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_80]

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_80]

at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_80]

at org.directwebremoting.impl.ExecuteAjaxFilter.doFilter(ExecuteAjaxFilter.java:34) [dwr.jar:]

at org.directwebremoting.impl.DefaultRemoter$1.doFilter(DefaultRemoter.java:428) [dwr.jar:]

at org.directwebremoting.impl.DefaultRemoter.execute(DefaultRemoter.java:431) [dwr.jar:]

at org.directwebremoting.impl.DefaultRemoter.execute(DefaultRemoter.java:283) [dwr.jar:]

at org.directwebremoting.servlet.PlainCallHandler.handle(PlainCallHandler.java:52) [dwr.jar:]

at org.directwebremoting.servlet.UrlProcessor.handle(UrlProcessor.java:101) [dwr.jar:]

at org.directwebremoting.servlet.DwrServlet.doPost(DwrServlet.java:146) [dwr.jar:]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.2.Final-redhat-1.jar:1.0.2.Final-redhat-1]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:295) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at com.kcs.ktam.web.filter.AuthorizationFilter.doFilter(AuthorizationFilter.java:123) [classes:]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at com.kcs.ktam.web.filter.DBSessionControlFilter.doFilter(DBSessionControlFilter.java:37) [classes:]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at com.kcs.ktam.web.filter.CrossScriptingFilter.doFilter(CrossScriptingFilter.java:29) [classes:]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.3.0.Final-redhat-14.jar:7.3.0.Final-redhat-14]

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:621) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:653) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.2.Final-redhat-1.jar:7.2.2.Final-redhat-1]

at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_80]

Best Regards,

Jongchai

[jaysensharma]

Database maintenance, network problems, or other outage events may cause JBoss EAP 6 to lose the connection to the database. You enable database connection validation using the <validation> element within the <datasource> section of the server configuration file. You can achieve the Fault Tolerance for the Database services  by enabling the some tags in your DataSource configuration as described in:

<validate-on-match>true</validate-on-match>
OR  
<background-validation>true</background-validation>

<valid-connection-checker class-name="classname.here.please.read.the.doc" />
<exception-sorter class-name="exception.sorter.classname.here.please.read.the.doc" />

https://access.redhat.com/documentation/en-US/JBoss_Enterprise_Application_Platform/6.4/html/Administration_and_Configuration_Guide/sect-Database_Connection_Validation.html

Enabling the above parameters will help JBoss EAP datasource to attempt to reconnect to the database as soon as the Database services will be restored.

NOTE: If the <validate-on-match> option is set to true, the <background-validation> option should be set to false. The reverse is also true. If the <background-validation> option is set to true, the <validate-on-match> option should be set to false.

Regards

Jay SenSharma

[jongchai]

Thank you very much.

I will inform team to test from your suggested.

Best Regards,

Somchai