Problems with message bus client subscriptions

[mtkimberlin]

So, I have been trying to get the Errai message bus working on it's own in an existing project. It's been a two steps forward, one step back kind of game so far.  Now I'm running into something that isn't quite clear to me.  After some digging, I came to realize something seems to be going wrong with the initialization of the message bus on the client.  Here's what I'm getting in the javascript console:

You can see that it seems to initially connect correctly (the phase=connection call returns normally and the response payload looks good from what I can tell).  However, after that I start getting 401s that are coming from the DefaultBlockingServlet from what I can tell.  I have tried serving that servlet from both within the Guice container and outside of it.  Both give the same result.  The JSON in the response from the 401 call says that the session is expired.  However, it is definitely not and other calls will proceed just fine.  I'm not sure why Errai is losing track of the session.  I also notice that once the first 401 happens there is an odd repetition of the "bus initialization started" and "wait for: org.jboss.bus.client.api.ClientMessageBus" messages.

Any thoughts on what might be causing this?

I should note that I'm using the message bus *only*...no CDI or any other part of Errai.

Any input would be greatly appreciated! Thanks!

[mtkimberlin]

So, I think this may be related to the transient nature of the queueSessions HashMap in the SessionsContainer in HttpSessionProvider.  We have a clustered environment that puts session attributes into memcache.

Can anyone tell me the driver behind making those transient?

[csa]

Hi Michael,

The console log seems to show the error in development mode too (not just in your cluster). Is this correct? Which version of Errai are you using?

Can you give us some details about your cluster environment: Are you using sticky sessions or session replication? The transient fields are by design as a QueueSession is specific to a single bus instance on the server. It wouldn't work if we serialized those and say replicated them to another cluster node as part of the HTTP session.

Cheers,

Christian

[mtkimberlin]

Christian,

Yes, it happens in dev mode...but our local environments utilize the same memcached session wrapper.  It's not sticky sessions...and it's not what you might normally think of when you think session replication.  Essentially, the session attributes are stuffed into memcached under a key that is built using the session id.  That id can then be used to recreate the key and pull attributes out of memcached in from any node in our cluster.  "Cluster" might even be a misleading term, as it likely brings to mind J2EE clustering of a kind that is not really applicable to what we are doing.  Our nodes are simple Jetty web nodes, and our applications are designed to be stateless wherever possible.  However, there are some limited situations where sessions are utilized to assist in caching some basic data and user authentication andauthorization states.  Basically, the request is wrapped to provide a session that knows how to talk to memcached to add and retrieve attributes.

I have a feeling that what is biting us here is that our sessions *prefer* memcached.  Which means, if they find an attribute there they will use it, because previous requests that alter our sessions could very well have been made to other nodes and we want the most recent data available.

How does this QueueSession play in a replicated session environment?  Wouldn't it have a similar issue to what I am experiencing here?

Thanks,

-michael

[csa]

Yes, when a client switches to a new cluster node it won't have a queue session on this node and will re-federate. If the HTTP session was replicated the new queue session should be associated with the same HTTP session, otherwise it will be associated with a new HTTP session.

So, seeing a 401 on the client once after switching to a new node is fine but this shouldn't be repetitive unless you're constantly switching to a new node that doesn't have a queue session for this client.

Are you seeing this repeating initialization after only one node change? If so, this would indicate a bug in Errai's message bus.

[mtkimberlin]

I see it after 0 node changes....that said, I disabled out session wrapper in the app and the problem goes away.  I'll see about possibly fixing that part in our wrapper.

Now the subscribe call goes through and returns 200, but when an event is sent for that subject I get:

org.jboss.errai.bus.client.api.base.NoSubscribersToDeliverTo: no subscribers to deliver to for subject: CustomerAlert3406000506

  at org.jboss.errai.bus.server.ServerMessageBusImpl.delayOrFail(ServerMessageBusImpl.java:294)

  at org.jboss.errai.bus.server.ServerMessageBusImpl.sendGlobal(ServerMessageBusImpl.java:261)

  at org.jboss.errai.bus.server.ServerMessageBusImpl$2.run(ServerMessageBusImpl.java:264)

  at org.jboss.errai.bus.server.ServerMessageBusImpl$3.run(ServerMessageBusImpl.java:309)

  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

  at java.util.concurrent.FutureTask.run(FutureTask.java:166)

  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)

  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)

  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

  at java.lang.Thread.run(Thread.java:722)

I did the obvious checking to make sure the subjects were the same.  Could I be running into different instances of the ServerMessageBusImpl between the subscribe call and the sending of the message perhaps?

Oh, and I never answered your question about version...we are using 2.4.2-Final

[mtkimberlin]

So, progress...I got past this issue by doing the following in a guice module:

    @Provides

    @Singleton

    protected ErraiService getErraiService() {

        if (ErraiServiceSingleton.isInitialized()) {

            return ErraiServiceSingleton.getService();

        }

        throw new IllegalStateException("ErraiService accessed before the singleton could be initialized, please make sure to use a Provider<ErraiService> and avoid access before servlet initialization.");

    }

    @Provides

    @Singleton

    protected RequestDispatcher getRequestDispatcher(ErraiService service) {

        return service.getDispatcher();

    }

I was definitely running up against multiple instances.  Now my subscription seems to be working as expected and the attempt at delivering the message finds the subscription, however I'm getting "class is not available to the marshaller framework".  I have annotated the classes in that hierarchy with @Portable.  Is there some other magic required that I'm not seeing?

Thanks again!

[mtkimberlin]

It was a missing ErraiApp.properties file that caused the class not available to the marshalling framework error. All is now right with the world.