5 Replies Latest reply on Feb 19, 2015 9:18 AM by ctomc

    Performance problems, wildfly 8.1, MDB application

    jasim

      Hello,

        I have deployed an mdb application in Wildfly 8.1, which reads data from Websphere MQ and, after some xml parsing accessing, putx the data into a remote Oracle database.

         (There are actually 3 schemas on that remote machine). The MQ Manager runs on the same machine as the wildfly process.

        

         What I observe is - the throughput of the application is rather poor. Only few messages per second. And more significantly, the load, the wildfly process causes, is enormous. The process

         uses 25% of CPU, on a AIX box with PowerPC Power7, 3.1 GHz, 4 Cores, 64Bit.

        

         I have doubled some pool sizes, like threads and mdb pool size, to improve the throughput but no effect.

        

         I really can't explain the load. I tried to get some statistics - these are only some observation I got

        

         1) Surprisingly for me, only one DB connection from 10 is active. Often, all connections are idle.

         2) Nearly all worker threads are in "parked" state. There is hardly any thread which does anything.

         3) There are some SLSB called by the MDB but I don't see any statistics for SLSB, e.g. pool-available-count is undefined. (why?)

        

         Would somebody be so kind and give some hints, where to look or guide me through the analysis?

        

      Best regards

        • 1. Re: Performance problems, wildfly 8.1, MDB application
          ctomc

          statistics need to be enabled.

           

          for messaging add <statistics-enabled>true</statistics-enabled> to messaging subsystem under <hornetq-server>

           

          and in ejb3 subsystem add  <statistics enabled="true" />

          • 2. Re: Performance problems, wildfly 8.1, MDB application
            jasim

            It appears that nearly all Runnable threads hang in a system code

             

            at sun/nio/ch/PollsetArrayWrapper.pollsetPoll(Native Method)
            at sun/nio/ch/PollsetArrayWrapper.poll(PollsetArrayWrapper.java:289)
            at sun/nio/ch/PollsetSelectorImpl.doSelect(PollsetSelectorImpl.java:73)
            at sun/nio/ch/SelectorImpl.lockAndDoSelect(SelectorImpl.java:99)
            at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:110)
            at sun/nio/ch/SelectorImpl.select(SelectorImpl.java:114)
            at org/xnio/nio/WorkerThread.run(WorkerThread.java:502)

             

            These threads are actually not application default-threads, but "default I/O" threads.

            What are these threads for? Can the number of I/O threads be reduced? Is it meaningful?

            • 3. Re: Performance problems, wildfly 8.1, MDB application
              ctomc

              Can you upgrade wildfly to 8.2 and what is the JDK you are using? any chance you can upgrade to latest 7 release?

              • 4. Re: Performance problems, wildfly 8.1, MDB application
                jasim

                The version is AIX specific

                 

                java version "1.7.0"

                Java(TM) SE Runtime Environment (build pap3270sr6-20131015_01(SR6))

                IBM J9 VM (build 2.6, JRE 1.7.0 AIX ppc-32 20131013_170512 (JIT enabled, AOT enabled)

                J9VM - R26_Java726_SR6_20131013_1510_B170512

                JIT  - r11.b05_20131003_47443

                GC   - R26_Java726_SR6_20131013_1510_B170512

                J9CL - 20131013_170512)

                JCL - 20131011_01 based on Oracle 7u45-b18

                 

                I requested an upgrade of java but it will take some time. We'll get the latest pap3270sr8fp10-20141219_01, 7.0.0.195, SR8 FP10 in March probably.


                Same for wildfly.


                It might be an AIX issue

                • 5. Re: Performance problems, wildfly 8.1, MDB application
                  ctomc

                  yeah I think it also might be JDK on AIX issue. or AIX itself.

                  It is hard to tell as we don't have any AIX to test on