-
46. Re:
"timfox" wrote:
The "right" way depends on what you are trying to do.
From your test program I can see you are just sending messages to a queue, and consuming them via an MDB which does nothing but log a message.
In this case you don't need transactions on your MDB, also there's no real JCA involved here, since you're just sending messages from a client - there's nothing to pool.
Also you are currently using persistent messages, for the purposes of this example that's unnecessary too.
Ah I see where you are coming from :-). My real application actually consists of two applications, the first accepts orders coming from http requests and the second processes the orders. Orders are transmitted from app1 to app2 over JMS. Both apps must persist the orders in their own databases. Earlier both the applications were simple web applications (war packaging), but when JMS was introduced in the mix, I switched to EAR because it seemed that MDBs/JCA/JTA is the "right" way to process JMS messages. This has obviously increased the complexity of the whole system and instead of focusing on the business functionality I am spending all my time on JMS. I will gladly move back to WAR if we can find a reasonable JMS approach without using MDBs. So that's what I am really trying to do (and I suspect that's what 90% of the people using JMS do anyway)!
In short, I need to persist objects on both systems. Both systems need to be able to send JMS messages to each other (orders and confirmations) - I have two separate queues set up for this - one in each direction. I can't loose any orders or confirmations - so I guess the messages need to be persistent.
The test program was just a simplification to figure out how the JMS piece should work. Hope this helps set up the context. -
49. Re:
"clebert.suconic@jboss.com" wrote:
"timfox" wrote:
Why are you persisting orders in a database?
If you are using persistent messages then the JMS system can guarantee persistence for you.
Usually an orders table needs to be kept for historical reasons, right?
That's a sweeping statement if ever I've heard one. -
51. Re:
Now not only are you doing that once you're doing that twice, so that will involve 2 JTA transactions.
Each JTA transactions will involve a minimum of 3 syncs to the file system, so that's 6 syncs +2 more for the databases = probably around 8 syncs.
So yes, it will be slow (and it's not because of JBM) ;) -
52. Re:
If we had transactions done per batch, we would solve not only the messaging bottleneck, but also the database bottleneck.
say... you could batch commits per 100 records. If 1 record failed you would eventually process the 100 records again what would give you a better throughput on both the database and messaging.
You still guarantee transaction on each message.
I' m almost sure this is what other companies are doing with SpecJ. -
54. Re:
"timfox" wrote:
Why are you persisting orders in a database?
To clarify, orders are persisted in two databases, one on each system. The two systems belong to different departments (or even different companies) - so there is an ownership issue here and both systems need to save the data. Moreover the database representation of an order is different on each system (different attributes, different relationships) based on their responsibilities. Only the common attributes needed by both systems are passed on the JMS message. I hope this clarifies the need for persisting the orders in the two databases."timfox" wrote:
If you are using persistent messages then the JMS system can guarantee persistence for you.
I am confused about this. I believe that JMS persists messages only to guarantee delivery in case the target system is down or not operating correctly. You do not expect me to look at persisted JMS messages as the system of record for my orders, do you? For me these are just text messages on a queue, not first class objects that can be associated with other objects in my database. Moreover, at least conceptually I think of persisted JMS messages to be deleted when the message has been consumed and acknowledged. So they are not a permanent record of my business transactions. Am I missing something?
Given these requirements, I know that I will need x milliseconds to store orders in my databases. All I am trying to do now is to make sure that JMS is minimum overhead on my system, which is a completely orthogonal issue. As I said earlier, I am prepared to embrace all the best practices on the JMS side to improve performance as long as I have reliable and performant messaging. I can punt MDBs, JTA, JCA - whatever you say as long as I have a decent alternative. So if you want me to reuse connections, sessions, consumers, producers, tell me how: write my own, use something provided by JBM, or pick something like Spring JMS template (with some connection pooling) and their MDP container! -
55. Re:
You do not expect me to look at persisted JMS messages as the system of record for my orders, do you?
This all depends on your business.
If the data should exist on the DB only for the life cycle of the message. (I mean, as soon as the message is consumed, the record is deleted), you could just store data on the message using a persistent message, and not use the DB for anything.
But if the data will exist on the DB after the message is processed, then you probably need the data on the DB for other business matters. But on that case the Database will be your bottleneck. -
56. Re:
"clebert.suconic@jboss.com" wrote:
You do not expect me to look at persisted JMS messages as the system of record for my orders, do you?
This all depends on your business.
If the data should exist on the DB only for the life cycle of the message. (I mean, as soon as the message is consumed, the record is deleted), you could just store data on the message using a persistent message, and not use the DB for anything.
But if the data will exist on the DB after the message is processed, then you probably need the data on the DB for other business matters. But on that case the Database will be your bottleneck.
+1 If you have a database in the picture, then more than likely that's the slowest point in the chain. JBM persistence is *extremely fast* (much faster than any database we've seen). If you're looking for performance gains your best bet is to optimise that database access. Like I said before, system performance is only as fast as the slowest link in the chain.
Regarding persistence in queues vs a database. Persistent messages in queues are just as persistent as messages in database, so it just depends what you prefer, Records in databases get deleted when the row is deleted, messages in queues get deleted when messages get acknowledged, but neither one is more or less persistent than the other.
In large companies where they want high throughput but also want to persist their messages in a database, a common pattern you will find will be to duplicate and route those messages to another persistent queue (see the chapter in the usermanual about diverts), then have some batch process that runs in the middle of the night which consumes the messages from that queue and stores them in a db in a batch fashion.
Messaging is extremely flexible and goes way beyond JMS. It all depends how you want to slice it. -
57. Re:
Clebert, Tim,
Thanks for your detailed responses. So I understand how to deal with database persistence. The only question that remains is how I should configure the JMS piece. Drop MDBs/JCA/JTA? (since that is turning out to be slow too.) But, replace it with what? My own connection/session/consumer/producer pools or some off-the-shelf framework?
Looking forward to your response.
Naresh -
58. Re:
It's very difficult for us to give a recommendation since we're not familiar with your business or architecture, and there's rarely a "one size fits all" solution.
But I'll try to give some pointers, if you're sending messages from web requests into an application in the same app server instance then using the JCA managed connection factory is the recommended approach (java:/JmsXA) and will give you the pooling you need. The overhead of this shouldn't be too high for sending messages.
Do you need MDBs? Well that really depends on whether you also need to persist the row to a database in the same transaction. Doing this in an MDB allows you to use JTA to do the message delivery and insert of row in the same global transaction. Of course this comes at an overhead.
If you can design out the insert into the DB that will give you your biggest performance boost, since then you can just use a standard JMS consumer and you can avoid JTA too. But as we've discussed before I'm not sure that's possible for you.
To demonstrate the overhead of all this stuff, try creating a simple program that uses raw JMS to send and consume from a queue. No JTA, MDB, JCA, Spring or anything else. And compare the performance.