5 Replies Latest reply on Jun 10, 2004 5:55 PM by praneel

    Strange Problem with HTTP Invoker Using Tomcat Coyote-HTTPS

    Arnold Hahamyan Newbie

      We are using JBoss 3.2.2/Tomcat 4.1.27 on RedHat 9. The client application (Swing) communicates with the app server via the HTTP invoker (HTTPS) on port 443. For testing we are using self-signed certs. We also use a custom JAAS login module for client authentication. The application deployed is an ear made up on stateless session beans using CMT against an Oracle datasource. We observed this problem only on Linux, it does not happen on Windows.

      The problem has to do with something in the http invoker trying to make calls out to java.sun.com and www.jboss.org to lookup dtds which causes the client invocation to hang. In our environment, outside access is not allowed from app server boxes. The server comes up fine without any problems, however the first time a client call is made, four dtds are requested (www.jboss.org and java.sun.com) , they are :

      /j2ee/dtd/jboss_3_2.dtd
      /j2ee/dtd/security_config.dtd
      /j2ee/dtd/jbosscmp-jdbc_3_2.dtd
      /j2ee/dtd/jbosscmp-jdbc_3_0.dtd
      /dtd/web-app_2_3.dtd

      If outside access is not allowed the client invocation hangs and netstat shows a SYN_SENT entry going out to the external site. About 15 minutes later the invocation seems to timeout, the client receives the expected response and things proceed fine thereafter, further client invocations do not hang. For now we modified etc/hosts to point www.jboss.org and java.sun.com to 127.0.0.1. This causes the invocation to succeed, however we observed that something on the app server tries to lookup the dtd by going to the external sites exactly every 5 minutes. I copied the dtds to an internal box running apache and pointed the www.jboss.org and java.sun.com to that machine and observed that even though the http response was a 200, it still tries to validate the dtds every five minutes. The same exact test on Windows worked with no problems, no http requests were observed from the app server to lookup any dtds.

      Basically, the first client invocation acts as a trigger to start the lookup of the five dtds every five minutes, regardless of whether they are found or not. Turning on debugging reveals the following exceptions which occur irregardless of client invocation hanging or running on windows/linux. The exceptions show up as DEBUG.

      Any help is greatly appreciated.

      Thanks,

      Arnold Hahamyan

      2004-06-04 11:03:14,484 DEBUG [org.apache.tomcat.util.threads.ThreadPool] Getting new thread data
      2004-06-04 11:03:14,781 DEBUG [org.apache.tomcat.util.net.jsse.JSSE14Support] Error getting client certs
      javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
      at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(DashoA6275)
      at org.apache.tomcat.util.net.jsse.JSSE14Support.getX509Certificates(JSSE14Support.java:151)
      at org.apache.tomcat.util.net.jsse.JSSESupport.getPeerCertificateChain(JSSESupport.java:161)
      at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:748)
      at org.apache.coyote.Response.action(Response.java:222)
      at org.apache.coyote.tomcat4.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:321)
      at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:221)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:601)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:392)
      at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:565)
      at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
      at java.lang.Thread.run(Thread.java:534)
      2004-06-04 11:03:14,812 INFO [org.jboss.security.plugins.JaasSecurityManagerService] Created securityMgr=org.jboss.security.plugins.JaasSecurityManager@1d1cdf7
      2004-06-04 11:03:14,812 DEBUG [org.jboss.security.plugins.JaasSecurityManager.http-invoker] CachePolicy set to: org.jboss.util.TimedCachePolicy@1292ba7
      2004-06-04 11:03:14,812 INFO [org.jboss.security.plugins.JaasSecurityManagerService] setCachePolicy, c=org.jboss.util.TimedCachePolicy@1292ba7
      2004-06-04 11:03:14,812 INFO [org.jboss.security.plugins.JaasSecurityManagerService] Added http-invoker, org.jboss.security.plugins.SecurityDomainContext@1f11e0f to map
      2004-06-04 11:03:15,156 DEBUG [org.apache.tomcat.util.net.jsse.JSSE14Support] Error getting client certs
      javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
      at com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificates(DashoA6275)
      at org.apache.tomcat.util.net.jsse.JSSE14Support.getX509Certificates(JSSE14Support.java:151)
      at org.apache.tomcat.util.net.jsse.JSSESupport.getPeerCertificateChain(JSSESupport.java:161)
      at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:748)
      at org.apache.coyote.Response.action(Response.java:222)
      at org.apache.coyote.tomcat4.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:321)
      at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:221)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:601)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:392)
      at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:565)
      at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
      at java.lang.Thread.run(Thread.java:534)
      2004-06-04 11:03:15,203 DEBUG [org.apache.coyote.http11.Http11Protocol] IOException reading request
      javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Software caused connection abort: socket write error
      at com.sun.net.ssl.internal.ssl.SSLSocketImpl.d(DashoA6275)
      at com.sun.net.ssl.internal.ssl.AppInputStream.available(DashoA6275)
      at org.apache.tomcat.util.net.TcpConnection.shutdownInput(TcpConnection.java:138)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:406)
      at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:565)
      at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:619)
      at java.lang.Thread.run(Thread.java:534)
      Caused by: javax.net.ssl.SSLException: java.net.SocketException: Software caused connection abort: socket write error
      at com.sun.net.ssl.internal.ssl.BaseSSLSocketImpl.a(DashoA6275)
      at com.sun.net.ssl.internal.ssl.SSLSocketImpl.a(DashoA6275)
      at com.sun.net.ssl.internal.ssl.SSLSocketImpl.a(DashoA6275)
      at com.sun.net.ssl.internal.ssl.SSLSocketImpl.a(DashoA6275)
      at com.sun.net.ssl.internal.ssl.AppOutputStream.write(DashoA6275)
      at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:668)
      at org.apache.coyote.http11.filters.ChunkedOutputFilter.end(ChunkedOutputFilter.java:206)
      at org.apache.coyote.http11.InternalOutputBuffer.endRequest(InternalOutputBuffer.java:379)
      at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:714)
      at org.apache.coyote.Response.action(Response.java:222)
      at org.apache.coyote.Response.finish(Response.java:343)
      at org.apache.coyote.tomcat4.OutputBuffer.close(OutputBuffer.java:326)
      at org.apache.coyote.tomcat4.CoyoteOutputStream.close(CoyoteOutputStream.java:125)
      at java.io.ObjectOutputStream$BlockDataOutputStream.close(ObjectOutputStream.java:1590)
      at java.io.ObjectOutputStream.close(ObjectOutputStream.java:662)
      at org.jboss.invocation.http.servlet.InvokerServlet.processRequest(InvokerServlet.java:140)
      at org.jboss.invocation.http.servlet.InvokerServlet.doPost(InvokerServlet.java:195)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
      at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
      at org.jboss.web.tomcat.security.JBossSecurityMgrRealm.invoke(JBossSecurityMgrRealm.java:220)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:494)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.java:246)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.jboss.web.tomcat.tc4.statistics.ContainerStatsValve.invoke(ContainerStatsValve.java:76)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
      at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2416)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
      at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:65)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:577)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
      at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
      at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
      at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
      at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
      at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:223)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:601)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:392)
      ... 3 more
      Caused by: java.net.SocketException: Software caused connection abort: socket write error
      at java.net.SocketOutputStream.socketWrite0(Native Method)
      at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
      at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
      at com.sun.net.ssl.internal.ssl.OutputRecord.a(DashoA6275)