11 Replies Latest reply on Oct 17, 2006 7:16 AM by marklittle

    Why don't JBoss Guys offer a solution to [TransactionImpl] T

    thof

      Hi,

      we developed an high throuhput app (a lot of DB insert/updates). Some customers are using jboss in production for that app. But sometimes jboss reports the well known (!!) but unfixed error (warning?)

      16:08:57,977 WARN [TransactionImpl] Transaction TransactionImpl:XidImpl[FormatId=257, GlobalId=actout04/1530891, BranchQual=, localId=1530891] timed out. status=STATUS_ACTIVE

      and the processing of records stops. Each record is processed in a seperate tx! There is no exception or any other hint to find the reason for the problem. If the tx times out, I would expect a rollback, but the processing seems to hang. What can we do to fix this problem, especially if we can't reproduce this behaviour?


      I'v started a Solaris truss for the hanging process and it delivers some strange infos:

      lwp_kill(661, SIGUSR2) = 0
      sigprocmask(SIG_SETMASK, 0x926FDAB0, 0x00000000) = 0
      setcontext(0x926FD960)
      Received signal #17, SIGUSR2 [caught]
      siginfo: SIGUSR2 pid=3401 uid=201 code=-1
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      sigprocmask(SIG_SETMASK, 0xFF38DB28, 0x00000000) = 0
      lwp_cond_wait(0x000376B8, 0x000376A0, 0xF99817E8) = 0
      lwp_cond_broadcast(0x000376B8) = 0
      lwp_mutex_wakeup(0x000376A0) = 0
      lwp_mutex_lock(0x000376A0) = 0
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x00000000) = 0
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      lwp_suspend(660) = 0 [0]
      lwp_kill(660, SIGLWP) = 0
      lwp_continue(660) = 0
      Received signal #33, SIGLWP [caught]
      siginfo: SIGLWP pid=3401 uid=201 code=-1
      lwp_mutex_lock(0xFF3940B0) = 0
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      setcontext(0x926FD960)
      sigprocmask(SIG_SETMASK, 0xFF38DB28, 0x00000000) = 0
      Incurred fault #1, FLTILL %pc = 0x00A192CC
      siginfo: SIGILL ILL_ILLTRP addr=0x00A192CC
      lwp_cond_broadcast(0x92801FE0) = 0
      Received signal #4, SIGILL [caught]
      siginfo: SIGILL ILL_ILLTRP addr=0x00A192CC
      lwp_cond_wait(0x92801FE0, 0xFF3940B0, 0x00000000) = 0
      lwp_mutex_wakeup(0xFF3940B0) = 0
      lwp_mutex_lock(0xFF3940B0) = 0
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      mprotect(0x00A193C0, 412, PROT_READ|PROT_WRITE|PROT_EXEC) Err#22 EINVAL
      lwp_sema_post(0x92801E78) = 0
      lwp_sema_wait(0x92801E78) = 0
      sigprocmask(SIG_SETMASK, 0xFF38DB28, 0x00000000) = 0
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x927FDA60) = 0
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x00000000) = 0
      sigprocmask(SIG_SETMASK, 0x927FDA60, 0x00000000) = 0
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x00000000) = 0
      setcontext(0x926FD960)
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x927FDA20) = 0
      lwp_kill(660, SIGUSR2) = 0
      sigprocmask(SIG_SETMASK, 0x927FDA20, 0x00000000) = 0
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x927FDAC0) = 0
      lwp_kill(660, SIGUSR2) = 0
      sigprocmask(SIG_SETMASK, 0x927FDAC0, 0x00000000) = 0
      setcontext(0x927FD970)
      Received signal #17, SIGUSR2 [caught]
      siginfo: SIGUSR2 pid=3401 uid=201 code=-1
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      sigprocmask(SIG_SETMASK, 0xFF38DB28, 0x00000000) = 0
      lwp_cond_wait(0x000376B8, 0x000376A0, 0xF99817E8) = 0

      lwp_cond_broadcast(0x000376B8) = 0
      lwp_mutex_lock(0x000376A0) = 0
      lwp_mutex_wakeup(0x000376A0) = 0
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x00000000) = 0
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      setcontext(0x927FD970)
      poll(0xF99817B8, 0, 1) = 0
      Incurred fault #1, FLTILL %pc = 0x00A194F0
      siginfo: SIGILL ILL_ILLTRP addr=0x00A194F0
      Received signal #4, SIGILL [caught]
      siginfo: SIGILL ILL_ILLTRP addr=0x00A194F0
      lwp_suspend(57) = 0 [169]
      lwp_mutex_wakeup(0xFF3940B0) = 0
      lwp_mutex_lock(0xFF3940B0) = 0
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      lwp_suspend(27) = 0 [169]
      sigprocmask(SIG_SETMASK, 0xFF38DB28, 0x00000000) = 0
      sigprocmask(SIG_SETMASK, 0xFF399870, 0x00000000) = 0
      lwp_suspend(14) = 0 [169]
      sysconfig(_CONFIG_SIGRT_MIN) = 38
      setcontext(0x927FD970)
      poll(0xF9981A68, 0, 1) = 0
      lwp_suspend(2021) = 0 [0]
      lwp_kill(2021, SIGLWP) = 0


      and so on. Is there a deadlock? So to use Jboss in production mode, I hope Jboss guys can offer a solution.

      Thx in advance

      Thomas

        • 1. Re: Why don't JBoss Guys offer a solution to [TransactionImp
          marklittle

          You want to use JBoss with transactions in a production mode? Are you using JBossTransactions? If not, then you should make the move. If you're not, then you should remember that the default transaction service does not support transaction recovery, which means you either don't have a need for recovery in your production deployment (which ultimately means you really don't need transactions within the application server and db transactions are sufficient), or you didn't realise.

          • 2. Re: Why don't JBoss Guys offer a solution to [TransactionImp
            thof

            We use the standard fast in-memory transaction manager for JBoss 4.0.2 as specified in conf/jboss-service.xml:

            <!--
            | The fast in-memory transaction manager.
            -->
            <mbean code="org.jboss.tm.TransactionManagerService"
            name="jboss:service=TransactionManager"
            xmbean-dd="resource:xmdesc/TransactionManagerService-xmbean.xml">
            ss
            300
            <!-- set to false to disable transaction demarcation over IIOP -->
            true
            <depends optional-attribute-name="XidFactory">jboss:service=XidFactory</de
            pends>


            The parameter "InterruptThreads" is not explicity set so it is false.

            • 3. Re: Why don't JBoss Guys offer a solution to [TransactionImp
              marklittle

              Fast in memory means no recovery. Is that what you really want?

              • 4. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                thof

                I know! But is the in-memory transaction manager service the cause for the tx time-out? If yes, which transaction manager service schould we use instead?

                So you don't recommend the default tx manager in production deployments?

                • 5. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                  marklittle

                   

                  "thof" wrote:
                  I know! But is the in-memory transaction manager service the cause for the tx time-out? If yes, which transaction manager service schould we use instead?


                  Install the local JTA version of JBossTS.


                  So you don't recommend the default tx manager in production deployments?


                  I would never recommend using transactions in deployment without recovery capabilities, no matter who provides the implementation. If you don't want recovery then chances are you don't want transactions either!

                  • 6. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                    thof

                    The app creates a lot of local DB transactions (CRUD) in parallel by using MDBs. So I expect from the standard tx manager a correct commit/rollback behaviour. But in our case we got this warning

                    WARN [TransactionImpl] Transaction TransactionImpl:XidImpl[FormatId=257, GlobalId=actout04/1530891, BranchQual=, localId=1530891] timed out. status=STATUS_ACTIVE

                    and from this point on no records are stored. BTW: Each record is stored in a separate TX by using REQUIRES_NEW for a session bean, which is called from the MBDs.
                    There is nothing to recover for the tx manager, if the tx is either commited or roll backed. The DB is in a consistent state. The key problem is: Why does JBoss/tx manager stop/suspend or whatever the current transaction?
                    Does the local JTA version of JBossTS solve this problem?

                    Thanks for your help, Mark.


                    • 7. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                      marklittle

                      Try installing the local JTA component go JBossTS.

                      • 8. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                        thof

                        Ok!

                        One last question: Can I install the arjuna TS for JBoss 4.0.2 or is 4.0.3SP1 required?

                        • 9. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                          marklittle

                          If you need 4.0.2 support then email support@arjuna.com. The version from JBoss is 4.0.3SP1 only, unfortunately.

                          • 10. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                            subramaniam.venkat

                            Hello mark,

                            I am using a session bean with the transaction attribute as Required.
                            In our application we get the same error

                            "WARN [org.jboss.tm.TransactionImpl] Transaction TransactionImpl:XidImpl [FormatId=257, GlobalId=vpsmp11a//214732, BranchQual=] timed out. status=STATUS_ACTIVE"

                            And after getting this exception our system hangs.

                            But when we restart Jboss every thing is fine and the records are processed ..


                            The output of truss from Solaris

                            lwp_mutex_lock(0x000E31F0) = 0
                            lwp_mutex_wakeup(0x000E31F0) = 0
                            lwp_mutex_lock(0x000E31F0) = 0
                            lwp_mutex_wakeup(0x000E31F0) = 0
                            lwp_mutex_wakeup(0x000E31F0) = 0
                            lwp_mutex_lock(0x000E31F0) = 0
                            lwp_mutex_wakeup(0x000E31F0) = 0
                            lwp_mutex_lock(0x000E31F0) = 0
                            lwp_mutex_wakeup(0x000E31F0) = 0
                            lwp_mutex_lock(0x000E31F0) = 0
                            lwp_mutex_wakeup(0x000E31F0) = 0
                            lwp_mutex_lock(0x000E31F0) = 0
                            lwp_mutex_wakeup(0x000E3458) = 0
                            lwp_mutex_wakeup(0x000E31F0) = 0
                            lwp_mutex_lock(0x000E3458) = 0
                            lwp_mutex_wakeup(0x000E3458) = 0

                            I am using the fast in memory transaction in deployment and i read your article about not using the fast in memory transaction in deployment.

                            But i have few question on the installation of the arjuna transaction service.

                            1. Should i have to change my code if i have to use arjuna transaction
                            2. I am trying to install arjuna transaction in windows xp and i am unsuccessful whether can i find a installation guide for window xp.
                            3. Is there a limit on the number transaction the fast in memory can handle
                            4. And i assume that by recovery you mean that JBoss will not go into a situation of complete hang.

                            Please i would be grateful if you can put some lite on the issues.

                            Thanks in advance,
                            Anand

                            • 11. Re: Why don't JBoss Guys offer a solution to [TransactionImp
                              marklittle

                              It turns out that Arjuna is unable to offer support on 4.0.2 now. You can try to back-port the code yourself and we would lend whatever assistance we can via the forum.