JmsConnectionPool: Allowing control over the number of Sessi

[tfennelly]

The current implementation of the JmsConnectionPool is designed to use a single JMS Connection and from it, create all JMS Session instances for the Session pool.

Not all JMS Providers support an unlimmited number of JMS Sessions per Connection. IBM Websphere MQ (WMQ) is one example if this, but there are probably others too, so we are treating this as a general issue with our JmsConnectionPool. Therefore, we need to modify the JmsConnectionPool code to support this.

I've been trying in vain to reproduce this on WMQ, so in an effort to make some sort of progress, I decided to forget about WMQ for now. I've reproduced the issue using mocks and we can come back to WMQ at a later date.

There are 2 general solutions as I see it...

Common to both solutions would be a pool of JMS Connections. For a provider that supports an "unlimited" (not to be taken literally ;) ) number of sessions/connection, this pool would ever only have a single Connection and all Sessions would be created from this.

Then, I think we can manage the relationship between this pool and the sessions created from its connections in one of two ways (remember - general approach):

1. Maintain a Session Pool per Connection on in the Connection Pool.
2. Maintain a single Session Pool for all Connections in the Connection Pool.

I would need to play with the code a little, but I think option #2 above might be the least intrusive on the JmsConnectionPool codebase. We can associate each Session with its owner Connection via the JmsSession class (part of the pooling code). Then we can manage adding sessions to the central pool by locating an existing Connection from the Connection Pool (one that can still create a Session i.e. has not hit its max session count), or by creating a new Connection in the Pool.

As I said... I really need to play with the code a little now, so this post is just to let people get an idea of what I'm thinking. If people have any comments, feel free to share them. Otherwise I'll post back here in a little while :)

[kconner]

It is probably best to go down the first suggestion.

One issue is that we are currently closing down sessions that are deemed unreliable. One example of this is an issue which is evident with JBoss Messaging, ironically using XASessions, where the XAResource appears to get into an inconsistent state and is, after that point, unusable. We have no real choice but to attempt to close down the session and create a new one, currently from the same connection.

In addition we need to track the connection/session association as any errors returned through the listener will force a closure of the connection and sessions, with sessions returning to the pool later being discarded rather than reused, and we will need to handle this without affecting the other sessions/connections in this pool.

It would also be a good idea to consider separating the XA/non-XA connections resources in case a provider enforces different limits per type, such as in the error you are trying to reproduce.

Kev

[tfennelly]

Thanks Kev.

Can you explain why you consider #1 to be a better option, over #2?

As I have it in my head at the moment (of course, I need to play with it in the code first), either option can accomodate for the issues you outline above.

At the end of the day, I don't see too much difference between either option. I just thought #2 might be less intrusive :)

[kconner]

Perhaps I have misunderstood what you are proposing in that case :)

The pool, as far as the client is concerned, should be a single entity but the internals are currently such that there is a single association between the connection and the sessions, whether XA or non XA.

This client view should be retained but it would be better for the internals of the pool to manage the associations between the connections and their sessions independently, both for non-XA and XA resources. The 'client pool' would therefore contain a number of internal pools.

Your second option sounds as if you intended to keep a single internal pool of the sessions but this would complicate the tracking of the relationships. Of course this may not be what you had intended.

Perhaps you can give an expanded description of your options so that we can get a better understanding of what you are proposing.

Kev

Kev

[kconner]

Oops, didn't mean to sign twice :)

[tfennelly]

Yeah... the name of the JmsConnectionPool class is probably confusing the situation a bit, since it's really pooling session.

Anyway... yes... as I see it, the JmsConnectionPool interface would remain as is. The mods would all be internal to the JmsConnectionPool class.

I think the Session->Connection relationship can be tracked easily in either case. I think it will be 6 of one half dozen of the other. With option #1 you'd have a slightly more complicated "free session" lookup because you'd need to look in multiple pools. With option #2 you'd have a slightly more complex lookup of all sessions associated with a connection (e.g. for connection cleanup) because they're all in one pool. In either case, it's not rocket science anyway.

I'll post back soon with more specific details.

[kconner]

Expanding on your proposals would be great, so we can see what you mean.

The first solution sounds the cleaner as, if I read it correctly, you are intending to separate the internal pooling for each connection. The second solution makes it sound as if you are only using a single pool for all sessions.

As for the name, it definitely does not represent what it does but it has always been that as far as I am aware. Not a good choice but not one I am responsible for :)

Kev

[tfennelly]

OK... I'm going with option #1 for now. I'm happy enough with that too.

So the current JmsConnectionPool maintains a relationship between a single connection and multiple sessions. So (termed loosely), what I think we need is a pool of JmsConnectionPools (as it's currently badly called :) ). OK... we can't do it exactly like that, so what I was thinking was to push the connection:sessions management code down into a new class called "JmsSessionPool".

Each JmsSessionPool will be responsible for maintaining the relationship between a single connection and its set of sessions (just like JmsConnectionPool is doing today). It would not be responsible for creating the connection (unlike todays JmsConnectionPool), but would be responsible for creating the sessions off a connection, cleaning them up the sessions etc.

Then, the roll of the JmsConnectionPool would change slightly in that it's now responsible for the management of a pool of JmsSessionPools. It would be responsible for creating the Connection instances and the JmsSessionPool associated with each instance.

When asked for a session, JmsConnectionPool would need to locate a JmsSessionPool that has a free session. If non are available, it will need to have the logic that decides whether or not a new JmsSessionPool needs to be created, or otherwise go into that "pool empty" loop.

Am I making sense?

[tfennelly]

Just noticed something that may be a bug in the current JmsConnectionPool code....

The handleCloseSession method returns the session to the "free" pool before releasing it's resources. Is that a potential bug?

[kconner]

"tfennelly" wrote:
The handleCloseSession method returns the session to the "free" pool before releasing it's resources. Is that a potential bug?

Looks like it could be. The session.releaseResources is intended to make a best effort attempt to close the associated resources, trapping any exceptions. At present it is only catching JMSExceptions thus opening up a possibility for the maps to be inconsistent for a while.

Create an issue against session.releaseResources and we can get it fixed.

[tfennelly]

Just to clarify... when I said...

"tfennelly" wrote:
The handleCloseSession method returns the session to the "free" pool before releasing it's resources. Is that a potential bug?

... what I meant was that this seems like a bug to me (JmsSession.releaseResources aside), because it seems to be happening at the wrong time. Shouldn't the resources be released/cleaned-up before the session is returned to the "free" pool? Shouldn't returning it to the pool be the last step in the process.

[kconner]

"tfennelly" wrote:
... what I meant was that this seems like a bug to me (JmsSession.releaseResources aside), because it seems to be happening at the wrong time. Shouldn't the resources be released/cleaned-up before the session is returned to the "free" pool? Shouldn't returning it to the pool be the last step in the process.

But the method is synchronized and, with the exception of the releaseResources behaviour, will be atomic.

Kev

[tfennelly]

Ah OK, that's true... I was just looking at the ordering of the logic and thought to myself ... that ain't right.

Either way, I would think it's better to have these things done in the "correct" order. The code is clearer etc. I don't like the idea of having things like that in the wrong order and relying on synchronization, behvior of other methods etc to compensate. That's another days discussion :)

[tfennelly]

So we went for a less intrusive implementation of this. Originally I was working on creating a new JmsSessionPool class and in it, collapsing those 4 Maps down to just a single Map and eliminating all the juggling of sessions between Maps. We decided not to do this at this point, so I then went back and just extended the existing "patterns" inside JmsConnectionPool to accommodate multiple connections.

So what we have right now (working - full CI build) is the old JmsConnectionPool with:

1. An inner JmsSessionPool class for managing sessions associated with a single connection. It basically contains code moved down from the JmsConnectionPool class.

2. The JmsConnectionPool has exactly the same interface as before. It now manages a list of JmsSessionPool objects (List). It gets a session by looping over the session pool list, asking each session pool for a session. If non of the existing pools have a free session, it will create a new connection and add a new session pool for that connection (if it hasn't hit "maxConnections").

"maxConnections" (i.e. max number of session pools) is currently calculated based on the existing "org.jboss.soa.esb.jms.connectionPool" config from the jbossesb-properties.xml file (which is actually the max total number of sessions for the whole connection pool), as well as a new config param called "max-sessions-per-connection", which is currently coming from the "poolKey" (i.e. jndiEnv) used to create the JmsConnectionPool.

So maxConnections = (maxSessions/maxSessionsPerConnection);

This is not quite right though. I don't think we can take maxSessionsPerConnection from the poolKey because of how the JmsConnectionPoolContainer works. Seems to me like this setting also needs to come from the jbossesb-properties.xml file, but we would need to have a setting per JMS Provider. We could base this on the initial context factory class name e.g.:

<property name="org.jboss.soa.esb.jms.max.sessions.per.connection_com.ibm.mq.jms.context.WMQInitialContextFactory" value="1" />

This is not ideal either because I think you may need a different value for max.sessions.per.connection on the same provider, depending on other params e.g. whether or not the connections are XA (ala this forum topic).

Not pretty :)

Options and opinions?

[tfennelly]

Or maybe the JNDI config (i.e. the "poolKey") is the best place to get max.sessions.per.connection from. Setting different values for max.sessions.per.connection on different JNDI configs (e.g. on a JMS Provider or on the JMSRouter) would result in a different poolKey and hence a different pool.

Am I reading this correctly?