HornetQ - Pooling resources is hard!

[monteirosp]

Hi there, guys!

I'm having a bad time trying to setup an application using JMS best practices, witch include connections, sessions, producers and consumers reuse through object pooling. The problem resides on the lack of methods to check and see if the objects residing inside the pool are still usable or not.

As a workarround for objects validation, I tried to invoke the Connection.start(), Connection.getMetaData(), and others, but none of them worked as expected (lauching exceptions only under certain conditions). I was thinking of creating objects like a message, or try to send a blank message to the server, but I see this as a memory leak and resource wasting...

So, any of you guys went through this problem before? Any ideas?

Thanks in advance!

[clebert.suconic]

I'm having a bad time trying to setup an application using JMS best practices, witch include connections, sessions, producers and consumers reuse through object pooling. The problem resides on the lack of methods to check and see if the objects residing inside the pool are still usable or not.

Pool, what pool?

You should use JCA for pooling on the server...

But preferably you should just reuse your objects accordingly.

Even if you reuse your connection... creating a consumer will involve a round trip call to the server.

[monteirosp]

The problem is that I'm developing standalone clients for HotnetQ (also in standalone mode), both producer and consumer rather than using an appserver  for them to live within. Is that a bad idea?

[clebert.suconic]

If you just have a standalone client. Just create the producer for the lifecycle of your application. And reuse it accordingly to your usecases.

I don't know the flow of your application, but you should probably identify a session means in terms of your application's conversation with the user, and keep the producers and consumers accordingly.

[monteirosp]

Ops, sorry... Guess there's some missing information! The object pooling works pretty good when the connection with the server remains stable. The problem occurs when the server crash.

The logic consists on a Tomcat server receiving http requests, processing some information and sending a message to Hornetq. That's the happy path.

When the server becomes unavailable, Tomcat keeps the unsent messages in a local queue, and when the server comes back to life, this local queue is consumed and sent to Hornetq server. That should also be a happy path if there were a way to purge the invalid pooled objects (sessions and message producers for now) and create new ones.

That's the big picture...

Thanks!

[clebert.suconic]

Why not configure the clients to reconnect?

http://hornetq.sourceforge.net/docs/hornetq-2.0.0.GA/user-manual/en/html/client-reconnection.html

[monteirosp]

I was trying to stick to the specification, but JMS seems to fail in several aspects, including those objects life cycle. Maybe it's time to try the HornetQ core.

Thanks for the tips!

Regards

[timfox]

I really don't understand what you're trying to do, and why you need to pool anything...

[clebert.suconic]

In HornetQ, JMS is a thin layer in top of core. The reconnect is available for the JMS layer also.

If you need client reconnects.. you can just configure them.

+1 to Tim.. I'm also confused in what you're trying to achieve.

Message was edited by: Clebert Suconic

[monteirosp]

By caching and reusing JMS objects, I'm trying to follow the advices posted here:

http://community.jboss.org/wiki/ShouldIcacheJMSconnectionsandJMSsessions

As I'm developing my queue clients (producers and consumers) to run outside a JEE container, I'm supposed to "to follow sensible JMS application design, and design your application to re-use JMS connections, sessions, consumers and producers". That's my point.

Dead horse: I can't count neither the server or the network connection are 100% stable, and from times to times one or both of them WILL fail. So the problem is that by the "re-use JMS connections, sessions, consumers and producers", when something goes wrong, the pure JMS API doesn't specify routines for reconnection and object consistency check, resulting in a pool (cache) full of invalid objects.

Also, you have already told me to try and use automatic reconnection features, available in the HornetQ API, but not in the pure JMS API. Is that correct or am I totally lost here?

[timfox]

Pooling is really only necessary where you're in a container like JEE or servlet.

If you're just writing a JMS client application outside the container, why would you need pooling?

"Sensible application design" doesn't mean pooling. It means just maintaining references to your connections, sessions, etc appropriately according to the semantics of your application.

[monteirosp]

By our design, every http connection request we receive in a certain servlet, will result in a message being posted to HornetQ server. It's a pretty high volume of requests, something around 1000 requests per minute. I believe there's an overhead of creating the objects responsible for handling the messaging, right? Shouldn't I reuse them? Is it ok to create a session and a producer for every single request I receive?

Thanks!

[timfox]

Ok, now you're talking about pooling inside the container. Earlier you said you wanted to pool *outside* the container

"As I'm developing my queue clients (producers and consumers) to run outside a JEE container,"

If you're really talking about pooling inside the servlet engine, you can just us the JCA adaptor as normal, which will do it's own caching.

[monteirosp]

Well, when I say outside the container, I mean it! It's just a Tomcat...

Now, is it possible to setup Tomcat to use JCA? Is it already embedded? Where can I get directions for it? I don't have much experience on that matter...

Updated: BTW, I have a couple of servers running Tomcat producing messages and a whole different one running HornetQ.

Thanks again!

[timfox]

If you mean the tomcat that comes with JBoss app server, then yes this is a container too and you can use JCA adaptor as normal.