JBoss / Hibernate / BMP (CMT) / DB2 / JDBC - DB locks are no

[balellu]

I am using

>>>JBoss-3.2.5
>>>Hibernate-2.0.3
>>>BMP with CMT EJB's - Using requires transaction attribute. EJB's use Hibernate
>>>DB2-8.1.7

I have a situation where Hibernate is used to create insert there records into DB and creating three entity beans for each record.

Once the above process is complete i am invoking a JDBC call to get some information from the same table where hibernate has inserted three new records. Once i try to execute the query through JDBC, app just hangs.

When i monitor the connections in the DB2 i can see lock waiting and when i kill the connection that is creating the lock then app resumes with exception.

Now i am trying to figure out where the problem could be? Can some one please help me with this.

Thanks.

[marklittle]

It definitely sounds like the transaction that scopes your original DB inserts hasn't completed or released the connection. I suggest you attach a debugger and monitor the initial creation session to see what happens.

[balellu]

Thanks for the reply. I agree with you, there could be some problem with transaction in DB inserts.

What do you mean by "suggest you attach a debugger and monitor the initial creation session to see what happens"?

If you want me to debug the app, then i have done it already and i did not find anything wrong. Code completely execute till it tries to execute the query using JDBC. App just stays here for ever and nothing happens, i dont get any error any where in the logs. This is making it tough to figure out the problem.

If i change my CMT attributes to "Supports" from "Required" then everything works fine. My app cannot run with "Supports" attribute.

My assumption is EJB container is not able to commit the DB inserts or not able to release the lock. In other words JTA transactions are hanging. Please let me know what you think. Also provide me with pointers to start working on.

Thanks

[marklittle]

I was thinking about a couple of things that a debug execution (step-by-step if necessary) might concentrate on:

(i) monitoring the connection to the datasource and checking that nothing is going awry there.

(ii) checking that the insert transactions within the application server do complete successfully.

[balellu]

Thanks for your reply. I have performed both the steps that you have mentioned below.

(i) I am using DB2 control center to check the open connections and connection details. There is no problem in the connection till the situation explained occurs.

(ii) I queried the DB after the 3- inserts are complete and it causes the lock.

In the database i can see that all the insert statements are creating two types of locks.
(a) Exclusive Lock (X) on the row where data is inserted
(b) Intention Exclusive Lock (IX) on the table

I also noted that all the inserts are causing locks. This means that none of the transactions are completed or i am not sure what is their status.

Does this mean that EJB container is able to start a transaction & acquire a lock? I dont understand what is happening after this step.

Is there a way where i can monitor what my EJB container is doing? So that i can exactly find out what is going wrong.

[balellu]

I was checking the locks at the DB end and i found out that

1. All inserts are made by the same connection
2. None of the data is getting inserted into the DB

This mean that commit is not happening. I have been working for almost 6 - days now and i need to crack this.

Thanks.

[marklittle]

"balellu" wrote:
I was checking the locks at the DB end and i found out that

1. All inserts are made by the same connection
2. None of the data is getting inserted into the DB

This mean that commit is not happening. I have been working for almost 6 - days now and i need to crack this.

Thanks.

So either commits aren't happening and the first transaction is still running when you come to do your queries (and that's what causes the blocking) or the first transaction is rolling back and something else is subsequently causing the blocking. The former gets my vote. Unfortunately I don't know how you might check that Hibernate has committed its transaction. It may be worth cross-posting to the Hibernate forum.

[balellu]

Thanks for the reply. I agree with you. I think commits are not happening. I have posted in Hibernate forums already and i did not get any response till now.

Can you help me to find out monitoring the EJB container? I turned on the flag to view call to container but it not of much help.

I at least must be able to figure out if the problem is with hibernate / JBoss or something else and substantiate it. Once we know where the problem is then we can find out the solution.

[marklittle]

Are you using XA transactions? If so, is it possible for you to create a dummy XAResource and register it with the initial Hibernate transaction? If you instrument the XAResource code (e.g., put print statements in it) then you should see your operations get called when that transaction commits. If you don't, then we know there is something wrong.

[balellu]

Mike, Thanks for the reply.

Are you using XA transactions? If so, is it possible for you to create a dummy XAResource and register it with the initial Hibernate transaction? If you instrument the XAResource code (e.g., put print statements in it) then you should see your operations get called when that transaction commits. If you don't, then we know there is something wrong.

We are not using XA.

Today after digging through the logs and debugging the app and looking at my DB connections at DB2 end. I figured out that JDBC transactions are not committed. Because of this when hibernate is trying to insert data in the table it just hangs.

All the above mentioned activities are performed in the same CMT. Please look at the below flow.

SLSB--->(CMT~Required)Transaction--->Invoke JDBC (Here locks to tables are held & not released)---->Entity Bean---->(CMT~Required) Transaction---->Invoke Hibernate (Now the app hangs here waiting for the locks to be released by the JDBC)

Now it seems that EJB container is not able to commit the JDBC transactions. Please let me know if my analysis is correct and also let me know how to fix this problem.

Thanks

[marklittle]

If the first transaction isn't committed then we've already established that that is the cause of the subsequent problem. But we still don't know why the first transaction isn't committed, do we?

The formatting of the forum doesn't help with the flow you gave. Would it be possible to re-do this so the format is preseved and show explicitly where the application server is involved alone and where Hibernate is involved?

Thanks.

[balellu]

After posting my response. I changed the flow as below.

SLSB--->Invoke JDBC (Perform Explicit commit)---->Entity Bean---->(CMT~Required) Transaction---->Invoke Hibernate (App works fine)

Now looking at this. I've got a question.

1. When a CMT starts container will commit transaction only when the method is about to complete. So that it can commit all the changes in one shot, Please confirm?

If answer to my above question is right then there is a problem with the code and not with JBoss or hibernate ot DB2. Plese validate.

If the problem is with the code. Then how do i solve it? I dont want to do the explicit commit of my JDBC transactions.

Please analyze my comments and provide me with directions.

Thanks

[marklittle]

So I didn't realise these invocations were happening in the same "chain".

Yes, the CMT will only commit (or rollback) when the method returns. You shouldn't commit it explicitly.

One way to solve this issue would be to get the hibernate invocation to use the same transaction as the CMT. However, I believe you said that you can't do that. True?

[balellu]

Mark,

So I didn't realise these invocations were happening in the same "chain".

Yes, the CMT will only commit (or rollback) when the method returns. You shouldn't commit it explicitly.

One way to solve this issue would be to get the hibernate invocation to use the same transaction as the CMT. However, I believe you said that you can't do that. True?

All these invocations are made in the same chain.

I will shrink my flows so that you get to see them:

Flow with deadlocks:
SLSB(CMT)-->JDBC(insert)-->EntityBean(CMT)-->Hibernate (insert)
In this flow JDBC is invoked with 'Required' CMT attribute so JDBC insert waits for container to commit and during this hibernate insert blocks the method completion and waits for JDBC to release locks. This creates the deadlock.


Flow with the fix:
SLSB(no CMT)-->JDBC(Explicit commit)-->EntityBean(CMT)-->Hibernate
In this flow JDBC is invoked with 'Supports' CMT attribute and JDBC insert is committed explicitly and during this hibernate insert is invoked with 'Required' CMT attribute since the previous JDBC transaction is explicitly committed hibernate insert succeeds.

I hope my post this time is clear. Let me know your comments

[marklittle]

If Hibernate uses the same transaction as the earlier JDBC operation, then (depending on the locking rules), it should inherit the same capabilities and be able to read the tables which are locked. But it would appear that is not happening. Can you confirm that the Hibernate insert is occurring in a new transaction?