Strange Errai error in the logs

[vwjugow]

Hi,

we have an Errai application running on Jboss 7.1. Last Friday we had to restart it and after that, we noticed that there was a new error in the logs:

2014/10/07 04:27:41,402 ERROR [stderr] (pool-32-thread-1) java.lang.IllegalStateException: Cannot reply.  Cannot find RequestDispatcher resource.

2014/10/07 04:27:41,402 ERROR [stderr] (pool-32-thread-1) at org.jboss.errai.bus.client.api.base.DefaultMessageBuilder$1.reply(DefaultMessageBuilder.java:124)

2014/10/07 04:27:41,402 ERROR [stderr] (pool-32-thread-1) at org.jboss.errai.bus.client.api.base.DefaultErrorCallback.error(DefaultErrorCallback.java:83)

2014/10/07 04:27:41,403 ERROR [stderr] (pool-32-thread-1) at org.jboss.errai.bus.client.api.base.DefaultErrorCallback.error(DefaultErrorCallback.java:35)

2014/10/07 04:27:41,403 ERROR [stderr] (pool-32-thread-1) at org.jboss.errai.bus.server.ServerMessageBusImpl.delayOrFail(ServerMessageBusImpl.java:305)

2014/10/07 04:27:41,403 ERROR [stderr] (pool-32-thread-1) at org.jboss.errai.bus.server.ServerMessageBusImpl.sendGlobal(ServerMessageBusImpl.java:270)

2014/10/07 04:27:41,404 ERROR [stderr] (pool-32-thread-1) at org.jboss.errai.bus.server.ServerMessageBusImpl$2.run(ServerMessageBusImpl.java:273)

2014/10/07 04:27:41,404 ERROR [stderr] (pool-32-thread-1) at org.jboss.errai.bus.server.ServerMessageBusImpl$3.run(ServerMessageBusImpl.java:318)

2014/10/07 04:27:41,404 ERROR [stderr] (pool-32-thread-1) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

2014/10/07 04:27:41,405 ERROR [stderr] (pool-32-thread-1) at java.util.concurrent.FutureTask.run(FutureTask.java:262)

2014/10/07 04:27:41,405 ERROR [stderr] (pool-32-thread-1) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)

2014/10/07 04:27:41,406 ERROR [stderr] (pool-32-thread-1) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)

2014/10/07 04:27:41,406 ERROR [stderr] (pool-32-thread-1) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

2014/10/07 04:27:41,406 ERROR [stderr] (pool-32-thread-1) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

2014/10/07 04:27:41,407 ERROR [stderr] (pool-32-thread-1) at java.lang.Thread.run(Thread.java:745)

This error is popping up in our logs periodically every few seconds, and we have never seen it before last Friday and we can't reproduce it in our other environments (this is happening in our production environment)

What could be the cause of this?

While googling we found the string 'Cannot reply. Cannot find RequestDispatcher resource.' in a xxxx..cache.html file (e.g schlawiner.phonegap/DA320A70BEB45CF04A75192EB5D4C58D.cache.html at master · hpehl/schlawiner.phonegap · GitHub) so that led us to question whether the problem here could have been that when we restarted the server, we had a client with the application opened in his browser, and when he regain connection after the restart, this error started to happen to him and somehow we are logging that in out logs.

This seemed like an interesting theory on Friday but since we're still seeing the error it's probably not the cause, but I thought it was worth mentioning it.

Thanks in advance,

Victor.-

[csa]

Hi Victor,

Yes, judging from the stack trace, your server is trying to send messages to a queue that is no longer being used (no client is listening on it). This shouldn't happen. It seems some clients didn't reconnect properly after your server restart. Which version of Errai are you using? Also, are you operating a cluster or only a single node?

Cheers,

Christian

[vwjugow]

Hi Christian, thanks for the response. Sorry I could not answer before. Also thank you for solving the issue, it's not happening any more  haha

Anyways, it would be nice if we understand this problem a little better in case we face it again.

We're using Errai 3.0.0.Final, and in single node.

When you say queue, it's that something related to how errai implements server-client communication?

Also wouldn't it be strange if the cause was a dead client, since it was still happening  after some days of having the server up, and redeploying?

V.-

[csa]

Hi,

We have made fixes to the client-side reconnection logic in 3.0.2.Final. So, it's definitely worth upgrading (also because refreshes in development mode will be a lot faster).

Yes, the server keeps a queue for every connected client (associated with the HTTP session). From your stack trace it looks like your client had a valid HTTP session but no queue which is why I suggested the reconnection failed for some clients.

It wouldn't necessarily be strange for this to go on for a while as users could leave their browsers open and Errai will try to reconnect indefinitely. It's hard to tell though without digging deeper (look at the client logs, look at the messages that couldn't be sent, etc.).

Cheers,

Christian

[vwjugow]

Thank you very much. We will definitely be upgrading to 3.0.2 and hopefully try 3.1.0-SNAPSHOT in the near future (need to upgrade server and gwt :/ )