Exception propagation using servlet transport

[ron_sigal]

Thread moved from http://www.jboss.com/index.html?module=bb&op=viewtopic&t=96861&start=20&postdays=postDays&postorder=postOrder&highlight=highlight on the developers forum.

[ron_sigal]

"DGuralnik" wrote:

It still throws java.lang.reflect.UndeclaredThrowableException. Jboss is 4.2.3 that contains remoting 2.2.2sp8

 <!-- Unified invoker (based on remoting) for invocations via HTTP with target EJB3 beans -->
 <mbean code="org.jboss.remoting.transport.Connector"
 name="jboss.remoting:service=connector,transport=servlet,target=ejb3"
 display-name="EJB3 Servlet transport Connector">
 <depends>jboss.aop:service=AspectDeployer</depends>
 <attribute name="Configuration">
<config>
 <invoker transport="servlet">
 <attribute name="dataType" isParam="true">invocation</attribute>
 <attribute name="timeout" isParam="true">600000</attribute>
 <attribute name="marshaller" isParam="true">org.jboss.invocation.unified.marshall.InvocationMarshaller</attribute>
 <attribute name="unmarshaller" isParam="true">org.jboss.invocation.unified.marshall.InvocationUnMarshaller</attribute>
 <attribute name="serverBindAddress">${jboss.bind.address}</attribute>
 <attribute name="serverBindPort">8080</attribute>
 <attribute name="path">unified-invoker/restricted/Ejb3ServerInvokerServlet</attribute>
 <attribute name="noThrowOnError" isParam="true">true</attribute>
 <attribute name="return-exception" isParam="true">true</attribute>
 </invoker>
 <handlers>
 <handler subsystem="AOP">org.jboss.aspects.remoting.AOPRemotingInvocationHandler</handler>
 </handlers>
 </config>

What is the Exception inside the UndeclaredThrowableException? Could you give a stacktrace?

[ron_sigal]

Prompted by Abbasi Ujjainwala, I've taken another look at this thread, and I think I see what the problem is. I'm guessing that you're working from the "EJBAndJNDIOverHTTPWithUnifiedInvoker" wiki page at http://www.jboss.org/community/docs/DOC-9632? The problem is, you're trying to do an EJB3 invocation, but you've added the "return-exception" parameter to a Connector configuration that is a blend of EJB2 and EJB3 artifacts. E.g., the InvocationMarshaller is strictly for EJB2.

Instead, you want to start with this this Connector configuration:

 <!-- Unified invoker (based on remoting) for invocations via HTTP with target EJB3 beans. -->
 <mbean code="org.jboss.remoting.transport.Connector"
 name="jboss.remoting:service=connector,transport=servlet,target=ejb3"
 display-name="EJB3 Servlet transport Connector">
 <depends>jboss.aop:service=AspectDeployer</depends>
 <attribute name="InvokerLocator">
 servlet://${jboss.bind.address}:8080/unified-invoker/Ejb3ServerInvokerServlet
 </attribute>
 <attribute name="Configuration">
 <handlers>
 <handler subsystem="AOP">org.jboss.aspects.remoting.AOPRemotingInvocationHandler</handler>
 </handlers>
 </attribute>
 </mbean>

and add the return-exception parameter:

<attribute name="InvokerLocator">
 servlet://${jboss.bind.address}:8080/unified-invok/Ejb3ServerInvokerServlet/?return-exception=true
</attribute>

Two other things.

1. noThrowOnError="true" probably isn't what you want. That setting will cause Remoting to return the exception as the value of the invocation, rather than throw the exception.

2. I gather that you've been experimenting with Remoting 2.5.0.SP1. Note that the treatment of exceptions has changed since the 2.2 releases. I didn't want to change the default behavior of Remoting in a minor release, so I made it possible to turn on the behavior of sending the actual exception from the server back to the client but left the default behavior alone. For release 2.4.0.GA and above, that behavior becomes the default, and it can be reverted to the original behavior by setting the parameter "dont-return-exception" to "true".

[seanmmcc]

Ron, I just made the change you describe in the jboss-service.xml and the corresponding url in the web.xml. I receive the same UndeclaredThrowableException. The change did not seem to have any impact. Can you please review the configuration below and let me know what I have missed.

web.xml

<servlet>
 <servlet-name>Ejb3ServerInvokerServlet</servlet-name>
 <description>The ServerInvokerServlet receives requests via HTTP
 protocol from within a web container and passes it onto the
 ServletServerInvoker for processing.
 </description>
 <servlet-class>org.jboss.remoting.transport.servlet.web.ServerInvokerServlet</servlet-class>
 <!-- Pass locatorUrl instead of invokerName because otherwise you end up
 sharing the same server invoker for org.jboss.invocation and org.jboss.aop
 type of invocations which you don't wanna do. Worth noting that invokerName
 is hardcoded and hence you cannot create a separate one that way, hence the
 use of locatorUrl. -->
 <init-param>
 <param-name>locatorUrl</param-name>
 <param-value>servlet://${firewall.address}:8080/unified-invoker/Ejb3ServerInvokerServlet/?return-exception=true</param-value>
 <description>The servlet server invoker</description>
 </init-param>
 <load-on-startup>1</load-on-startup>
 </servlet>

jboss-service.xml

 <!-- Unified invoker (based on remoting) for invocations via HTTP with target EJB3 beans. -->
 <mbean code="org.jboss.remoting.transport.Connector"
 name="jboss.remoting:service=connector,transport=servlet,target=ejb3"
 display-name="EJB3 Servlet transport Connector">
 <depends>jboss.aop:service=AspectDeployer</depends>
 <attribute name="InvokerLocator">servlet://${firewall.address}:8080/unified-invoker/Ejb3ServerInvokerServlet/?return-exception=true</attribute>
 <attribute name="Configuration">
 <handlers>
 <handler subsystem="AOP">org.jboss.aspects.remoting.AOPRemotingInvocationHandler</handler>
 </handlers>
 </attribute>
 </mbean>

[ron_sigal]

Good catch with web.xml - I forgot about that one. I don't see anything wrong with your configuration.

Hmm. Something should have changed, since there should be a proper EJB3 Connector where there wasn't one before. I gather that your test program is

public void testRemoteBean() throws Exception {
 InitialContext ctx = null;
 EmarVersion versionBean = null;
 try {
 Properties env = new Properties();
 env.setProperty(Context.INITIAL_CONTEXT_FACTORY,
"org.jboss.naming.HttpNamingContextFactory");
 env.put("java.naming.provider.url",
"http://localhost:8080/unified-invoker/JNDIFactory/?return-exception=true");
 env.put("java.naming.factory.url.pkgs",
"org.jboss.naming:org.jnp.interfaces");

 ctx = new InitialContext(env);
 versionBean = (EmarVersion) ctx.lookup("ejb/remote/EmarVersion-http");

 String allReleaseNotes = versionBean.getAllReleaseNotes(false);
 System.out.println(allReleaseNotes);
 } catch (Exception e) {
 e.printStackTrace();
 }
 }

So, the exception is coming from

String allReleaseNotes = versionBean.getAllReleaseNotes(false);

right? I mean, the JNDI lookup is returning a proxy?

What I'm getting at is, are you really connecting to the correct Remoting Connector? Can we verify that the EmarVersion EJB3 is receiving the invocation?

If so, then what's inside the UndeclaredThrowableException? If the problem is just a matter of not properly configuring the exception throwing behavior, then I would expect to see a WebServerError with the error message in the exception thrown by the EmarVersion invocation.

[seanmmcc]

My test is almost identical and dies at the spot you reference. I am being returned a proxy. Heres the exception on the client:

java.lang.reflect.UndeclaredThrowableException
 at $Proxy1.isAuthenticatedUsingEncryptedPassword(Unknown Source)
 at com.v2.ejb.TestRemoteBean.main(TestRemoteBean.java:53)
Caused by: org.jboss.remoting.transport.http.WebServerError: <html><head><title>JBossWeb/2.0.1.GA - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - Error occurred processing invocation request. </h1><HR size="1" noshade="noshade"><p><b>type</b> Status report</p><p><b>message</b> <u>Error occurred processing invocation request. </u></p><p><b>description</b> <u>The server encountered an internal error (Error occurred processing invocation request. ) that prevented it from fulfilling this request.</u></p><HR size="1" noshade="noshade"><h3>JBossWeb/2.0.1.GA</h3></body></html>
 at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:405)
 at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:136)
 at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
 at org.jboss.remoting.Client.invoke(Client.java:1634)
 at org.jboss.remoting.Client.invoke(Client.java:548)
 at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:62)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.aspects.security.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:53)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:74)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.ejb3.stateless.StatelessRemoteProxy.invoke(StatelessRemoteProxy.java:107)
 at $Proxy1.isAuthenticatedUsingEncryptedPassword(Unknown Source)
 at com.artromick.emar.v2.ejb.TestRemoteBean.main(TestRemoteBean.java:53)
 at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:74)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.aspects.tx.ClientTxPropagationInterceptor.invoke(ClientTxPropagationInterceptor.java:61)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.aspects.security.SecurityClientInterceptor.invoke(SecurityClientInterceptor.java:53)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:74)
 at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
 at org.jboss.ejb3.stateless.StatelessRemoteProxy.invoke(StatelessRemoteProxy.java:107)

[ron_sigal]

OK, that "Error occurred processing invocation request." is coming from Remoting's ServletServerInvoker, so that's good. It doesn't prove that you got to the *right* instance, though, or that you got as far as the actual EJB3. Do you see any evidence that the EmarVersion has been called?

When ServletServerInvoker gets an exception back from AOPRemotingInvocationHandler (which tries to call the EJB3), it logs the message at DEBUG level:

 catch(Throwable ex)
 {
 log.debug("Error thrown calling invoke on server invoker.", ex);

Do you see anything like that in the server log?

[seanmmcc]

Ron,
I set my log level to debug like so (I see no new information in my server log):

<category name="org.jboss.remoting">

 <priority value="DEBUG" />

 </category>

 <category name="org.jboss.aop">

 <priority value="DEBUG" />

 </category>

I have been testing by setting a break point on the server and confirming the exception is exercised:

if (results.size() == 0) {
 throw new RemoteException(" Failed to find user with name supplied ");
 }

If I remove fix the invalid user name I am testing with the everything works as expected. So I think the round trip is working correctly.

Another developer tested and produced the same results I have described.

[dpfelten]

Ron -

I'm also receiving this error (and working with Sean on the problem). We can definitely verify that the EJB3 is being called - by using a breakpoint, we can stop before the RemoteException gets thrown, then step the whole way down the stack until we get to ServletServerInvoker.

The code that gets invoked in ServletServerInvoker is the following:

try
 {
 // call transport on the subclass, get the result to handback
 invocationResponse = invoke(invocationRequest);
 }
 catch(Throwable ex)
 {
 log.debug("Error thrown calling invoke on server invoker.", ex);
 invocationResponse = null;
 response.sendError(500, "Error processing invocation request. " + ex.getMessage());
 }

What should happen, from what I gather at least, is that the invocationResponse should be a marshalled exception that's then returned to the client. But in org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke, it catches an exception on newSi.invokeNext() and tries to marshal the exception. However, during the marshalling of the exception, it sees that it isn't a local call so it just re-throws the same exception. See here:

public static InvocationResponse marshallException(Invocation invocation, Throwable exception, Map responseContext) throws Throwable
 {
 if (invocation.getMetaData(IsLocalInterceptor.IS_LOCAL,IsLocalInterceptor.IS_LOCAL) == null) throw exception;

 InvocationResponse response = new InvocationResponse();
 response.setContextInfo(responseContext);

 response.addAttachment(IsLocalInterceptor.IS_LOCAL_EXCEPTION, new MarshalledObjectForLocalCalls(exception));

 return response;
 }

The if statement is true, so it throws the exception, which is caught in ServletServerInvoker (in the code sample above). It then rolls over the exception and sends back a 500 error with the given error message.

On the client, we can see in HTTPClientInvoker.useHttpURLConnection, this condition happening:

if(result instanceof String)
 {
 // this is a html error page displayed by web server, need to conver to exception
 WebServerError ex = new WebServerError((String)result);
 return ex;
 }

The result is the HTML as specified in the error Sean pointed out earlier in the thread, and the local variable metadata contains the headers including the 500 return code, the error message from the ServletServerInvoker, etc.

Hope this helps narrow things down for you. At this point, I think we need either a different configuration to avoid the code in question to modify the code to handle exceptions. Or something you suggest :-)

[abbasi]

Ron, one loud thought can we configure to use one of the InvokerServlet which i think are used internally by jboss console?
in their processRequest looks like they wrap the exception in MarshalledValue and write to out stream back.

[ron_sigal]

Hi guys,

"seanmmcc" wrote:

So I think the round trip is working correctly.

Cool. Like Lenny says on "Law and Order", "Sorry, we have to ask." :)

"dpfelten" wrote:

The code that gets invoked in ServletServerInvoker is the following:

try
 {
 // call transport on the subclass, get the result to handback
 invocationResponse = invoke(invocationRequest);
 }
 catch(Throwable ex)
 {
 log.debug("Error thrown calling invoke on server invoker.", ex);
 invocationResponse = null;
 response.sendError(500, "Error processing invocation request. " + ex.getMessage());
 }

Dave, are you sure about that? You're showing the ServletServerInvoker method

 public void processRequest(HttpServletRequest request, HttpServletResponse response)
 throws ServletException, IOException

but, in fact, that method is never called in Remoting. Instead, org.jboss.remoting.transport.servlet.web.ServerInvokerServlet.processRequest() calls the ServletServerInvoker method

 public byte[] processRequest(HttpServletRequest request, byte[] requestByte,
 HttpServletResponse response)
 throws ServletException, IOException

which has the code

 try
 {
 // call transport on the subclass, get the result to handback
 invocationResponse = invoke(invocationRequest);
 }
 catch(Throwable ex)
 {
 log.debug("Error thrown calling invoke on server invoker.", ex);
 invocationResponse = ex;

 if (checkForExceptionReturn(metadata))
 {
 String sessionId = invocationRequest.getSessionId();
 ServletThrowable st = new ServletThrowable(ex);
 invocationResponse = new InvocationResponse(sessionId, st, true, null);
 }
 else
 {
 isError = true;
 }
 }

Here, checkForExceptionReturn() looks for the presence of the "return-exception" parameter in the metadata map. I had to reread the code for a while to figure out how that parameter gets into the metadata map, and it happens in the lines

 Map urlParams = request.getParameterMap();
 metadata.putAll(urlParams);

where ServletRequest.getParameterMap() gets parameters from the query string, at the end of the InvokerLocator. If you look at the rest of ServletServerInvoker, you'll see that the InvocationResponse wrapping the ServletThrowable gets returned as a normal response.

On the other hand, if "return-exception" ISN'T set to true, you would get the variable "isError" set to true, which eventually results in

 if(isError)
 {
 response.sendError(500, "Error occurred processing invocation request. ");
 }

getting executed. So, I think that's the first thing to look for. In particular, check the client-bind-url entry in your jboss.xml file.

Now, on the client side,

"dpfelten" wrote:

On the client, we can see in HTTPClientInvoker.useHttpURLConnection, this condition happening:

if(result instanceof String)
 {
 // this is a html error page displayed by web server, need to conver to excepti
on
 WebServerError ex = new WebServerError((String)result);
 return ex;
 }

is strange. This code should be accessible only if "NoThrowOnError" is set to "true" in the invocation metadata map, which shouldn't be happening. However, make sure that "NoThrowOnError" is removed.

Ok, I've been testing Remoting and it looks like, if things are properly configured, the exception throwing mechanism works the way it should. So I think we're back to figuring out what's wrong with the configuration.

To be continued ...

[dpfelten]

Ron -

I went by memory in my initial reply, so I got a couple details wrong - sorry, it was the weekend :-). (Birthday weekend, no less!)

That said, you're right about the processRequest method that's called. The source code that gets executed for us is the following:

try
 {
 // call transport on the subclass, get the result to handback
 invocationResponse = invoke(invocationRequest);
 }
 catch(Throwable ex)
 {
 log.debug("Error thrown calling invoke on server invoker.", ex);
 invocationResponse = ex;
 isError = true;
 }

I downloaded the source code for 2.2.2.SP1 and see what you posted, with the checkForExceptionReturn(metadata) method call. When I attach the debugger to that source, however, the checkForExceptionReturn method is never invoked - isError is set immediately. Attaching the debugger with code from JBossRemoting-2_2_0_GA partially lines up (at least in behavior if not exact line numbers while debugging). See [1] for a screenshot that includes the return-exception set to true (in the variables debug window). Even though it's true, it never gets returned.

The moral of the story: we seem to have an outdated version of JBoss both on our development machines (4.2.2 GA) and in production (JBoss-AS 4.3 sans patches). Given the code you showed us, it should indeed fix the issue.

Regarding the WebServerError, "NoThrowOnError" isn't set anywhere that I can see. I think before we spend any more of anybody's time, we should upgrade all our servers to the latest patch of JBoss-AS 4.3. I just downloaded jboss-eap-4.3.0.GA_CP03.zip via the link provided in the service ticket. I poked around a bit looking for the source, but couldn't find it -- is it available, and if so, could you send a link?

Thanks for your help Ron.

Dave

[1] - http://www.seinberg.net/images/stack.png

[dpfelten]

Scratch that last question - I found the source!