1 2 3 Previous Next 37 Replies Latest reply on May 7, 2007 11:26 AM by fhh

    10,000 Recorc Per Second (In EJB 3.0)

    paata lominadze Master

      hi all,
      i have some mission critical tasks into my project, is it possible to persist 10 000 record per seconds,

      1. AS - JBoss Application Server 4.0.4GA
      2. Database - Oracle 10G 10.2.0.1
      3.EJB - 3.0 Framework
      4.OS - SunOS 5.10
      4.Server - Memory: 16G phys mem, 31G swap, 16 CPU,

      i know that i need performace
      here is my configurations about performance

      1. JVM Config Into JBoss

      JAVA_OPTS="-server -Xmx3168m -Xms2144m -Xmn1g -Xss256k -d64 -XX:PermSize=128m -XX:MaxPermSize=256m
       -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000
       -XX:ParallelGCThreads=20 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
       -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:+AggressiveOpts
       -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintTenuringDistribution
      


      2. Also i have one simple Stlateless Session Bean
      @Stateless
      @Remote(UsageFasade.class)
      public class UsageFasadeBean implements UsageFasade {
      
       @PersistenceContext(unitName = "CustomerCareOracle")
       private EntityManager oracleManager;
       @TransactionAttribute(TransactionAttributeType.REQUIRED)
       public long createUsage(UsageObject usageObject, UserContext context)
       throws UserManagerException, CCareException {
       try {
       oracleManager
       .createNativeQuery("INSERT INTO USAGE "
       + " (ID, SESSION_ID, SUBSCRIBER_ID, RECDATE, STARTDATE, APPLIEDVERSION_ID, CHARGINGPROFILE_ID, TOTALTIME, TOTALUNITS, IDENTIFIERTYPE_ID, IDENTIFIER, PARTNO, CALLTYPE_ID, USAGETYPE, APARTY, BPARTY, CPARTY, IMEI, SPECIFICCALLTYPE, APN, SOURCELOCATION, SMSCADDRESS, MSC_ID, ENDREASON, USAGEORIGIN, BILL_ID, CONTRACT_ID) "
       + " VALUES(SEQ_USAGE_ID.NEXTVAL, NULL, NULL, SYSDATE, SYSDATE, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL) ");
       return 1;
       } catch (Exception e) {
      
       }
       }
      }
      


      3. and into client side i have 200 Threads, each of them tried to call this method 50 times
      my result is that i can persist 10000 record in 20 seconds, without hibernate, with hibernate i got worst result :(,

      also i hear that it is good idea to use JDBC 3.0 driver for performance,
      i download newest oracle jdbc jar file from oracle site
      http://www.oracle.com/technology/software/tech/java/sqlj_jdbc/htdocs/jdbc_10201.html

      is this jar file JDBC 3.0 driver ?

      is it any more performance tuning into JBoss or EJB with entity beans?

      can anybody help me ? or is there any doc which can help me ?

      Regards,
      Paata,



        • 1. Re: 10,000 Recorc Per Second (In EJB 3.0)
          fhh Expert

          If all you want to do is to insert 10.000 meaningless records than write a stored procedure.

          If not - well, tell us what you want to do.

          Regards

          felix

          • 2. Re: 10,000 Recorc Per Second (In EJB 3.0)
            Elias Ross Master

            Let me just say that you first need to understand what it takes to get Oracle to insert 10,000 records per second.

            • 3. Re: 10,000 Recorc Per Second (In EJB 3.0)
              fhh Expert

              Insering 10.000 rows into an empty table is not that difficult. Any Oracle database should be able to handle that.

              Regards

              Felix

              • 4. Re: 10,000 Recorc Per Second (In EJB 3.0)
                Oskar Carlstedt Novice

                Hi!!

                Using EJBs here is probably not the best way to go. Doing so will cause the container to create a lot of insert statements - too many to get the performance you want. If I'm wrong here, let me know.

                I did something like this a couple of years ago with Oracle 8.1.6. If you don't care about all transactions a good way might be to create a data file (normally a comma seprataed flat file), which you later do an import on. Here you get performance, but if it is an good design. I don't think so. Good design and performance are often something that is hard to combine.

                Another way is to create a stored procedure that you call when you have, let say 5000 records to insert.

                One thing to think of is the data transport. Using Oracle from Java you go from client(s) to a java server, to PL/SQL to Oracle DB.

                Kind Regards
                Oskar

                • 5. Re: 10,000 Recorc Per Second (In EJB 3.0)
                  Oskar Carlstedt Novice

                  ...

                  one more thing. Whater decision you make here. Make sure you are using the native Oracle driver.

                  //Oskar

                  • 6. Re: 10,000 Recorc Per Second (In EJB 3.0)
                    paata lominadze Master

                    hi all,
                    great tnx for your posts,

                    yesterday i tried to count real amount of records and i got i need 2000 operation in one second, (operation means some selects and 5 insert into separate tables)

                    fhh


                    If all you want to do is to insert 10.000 meaningless records than write a stored procedure.
                    If not - well, tell us what you want to do.


                    i thought on this idea, but i don't need business logic into database, it is business request.

                    genman

                    Let me just say that you first need to understand what it takes to get Oracle to insert 10,000 records per second


                    i tried to insert 10 000 record into oracle database from pl/sql developer over network and it takes 5 minutes.

                    oskar.carlstedt Post

                    Using EJBs here is probably not the best way to go. Doing so will cause the container to create a lot of insert statements - too many to get the performance you want. If I'm wrong here, let me know.


                    you are right, it is not correct way to persist 2000 record using entity beans and hibernate,

                    but i thought, maybe is there any performance configuration which can increase performance,


                    Make sure you are using the native Oracle driver


                    http://www.oracle.com/technology/software/tech/java/sqlj_jdbc/htdocs/jdbc_10201.html


                    • 7. Re: 10,000 Recorc Per Second (In EJB 3.0)
                      Oskar Carlstedt Novice

                      It is possible to configure hibernate to do bulk operation. Perhaps you've already tried it? If not, take a look a http://www.hibernate.org/hib_docs/v3/reference/en/html_single/#queryhql-bulk

                      //Oskar

                      • 8. Re: 10,000 Recorc Per Second (In EJB 3.0)
                        fhh Expert

                         


                        i thought on this idea, but i don't need business logic into database, it is business request.


                        I strongly disagree: All business logic that can convienently handled by the database should be handled by the database. It is a zillion times faster, easier to implement and easier to maintain. There is no point in moving data in and out of the database for nothing.

                        If you think that is my idiosyncratic position read the following statement from Gavin King:

                        The main example of where ORM - and indeed Java - is not suitable is the case of processing data in bulk. It is simply never going to be efficient to fetch millions of rows from the database, into your JVM, and then update them one at a time. Don't use Java for this. Use a stored procedure.

                        http://www.javaperformancetuning.com/news/interview041.shtml

                        Regards

                        Felix

                        • 9. Re: 10,000 Recorc Per Second (In EJB 3.0)
                          Kestas ruks Newbie

                          Hi,
                          Few month ago I made some test and achieved more than 5000/s inserts on average pc (actually there was 2 pc's in case with oracle, one with jboss another with oracleand 1 pcin case with postgresql, both resutls quite equal). These inserts was simple, dummy inserts.
                          It's importat to set batch updates, can be 1000 or more


                          very important to switch off logging


                          use oci driver (in case with oracle)

                          also i used few threads (best speed with 8 threads, i thing this depends on pc (how many processors and so on)), everything was done i one transaction, also you can play with flush().

                          • 10. Re: 10,000 Recorc Per Second (In EJB 3.0)
                            paata lominadze Master

                            hi snau2005,


                            very important to switch off logging

                            of course i did it.

                            use oci driver (in case with oracle)

                            is it helpfull ?? do you think that oci driver is so faster then thin driver ?


                            also i used few threads (best speed with 8 threads, i thing this depends on pc (how many processors and so on)), everything was done i one transaction, also you can play with flush().


                            of course i use Thread, Thread will be created on client side and all of them calls my stateless session beam method, i tried to create 50 Threads, and my pc is enough strong, it has 16 CPU, and 16 GB Memory, Sun Solaris,

                            is it good idea that i use oracle cluster or JBoss Cluster ?

                            and i cant found any documentation about stored procedures into ejb 3.0, can anybody show me it ?



                            Regards,
                            Paata.



                            • 11. Re: 10,000 Recorc Per Second (In EJB 3.0)
                              Kestas ruks Newbie

                               

                              use oci driver (in case with oracle)

                              is it helpfull ?? do you think that oci driver is so faster then thin driver ?

                              I'm not sure in case of perfomance (thin driver had some bugs and i switched to oci (but this was long ago, now maybe thin also ok)

                              of course i use Thread, Thread will be created on client side and all of them calls my stateless session beam method, i tried to create 50 Threads, and my pc is enough strong, it has 16 CPU, and 16 GB Memory, Sun Solaris,

                              As I unterstood, every client Thread creates (calls ejb witch creates TR) his own transaction? so in reality you have 10000 transactions with one insert statement, thats of cource i think should be slow. In my case I called only one time ejb , so 1 TR was created then 8 Threads started to persist and flush.


                              and i cant found any documentation about stored procedures into ejb 3.0, can anybody show me it ?


                              Example to call stored function
                              try {
                               HibernateEntityManager hem = ((HibernateEntityManager)em.getDelegate());
                               Session session = hem.getSession();
                               Connection connection = session.connection();
                              
                               CallableStatement proc = connection.prepareCall("{ ? = call getNBVZSInfo(?, ?) }");
                               proc.registerOutParameter(1, Types.VARCHAR);
                               proc.setString(2, traderCode);
                               proc.setString(3, userName);
                               proc.execute();
                               result = proc.getString(1);
                              
                               log.info("Info: " + result);
                               } catch (Exception e) {
                               log.error("getNBVZSInfo, error: " + e);
                               }


                              and oracle functions looks like:
                              FUNCTION getNBVZSInfo(traderCode varchar2, userName varchar2) RETURN varchar2 is
                               rez varchar2(4000);
                               begin
                              
                               rez := 'hoho and the bottle of rum';
                               return rez;
                              end;


                              But again if bussines logic alows procedure to store 10000 , so one ejb call also can store 10000 in one call and if you will call oracle procedure 10000 time thats also will be slow.




                              • 12. Re: 10,000 Recorc Per Second (In EJB 3.0)
                                Oskar Carlstedt Novice

                                Hi again!

                                Here is some information from a project I attended a couple of years ago. We operated on a Sun Starfire (Sun Enterprise 10000), totally 32 cpus and 32GB RAM, our domain had 12 cpus and 12 GB RAM. A lot has hapend since then, but still, such computers are not fast when it comes to simple things like database inserts. They are used for extreme load and may calulate a lot of complex stuff.

                                We hade Oracle on the starfire. On another server we had a Java program doing import stuff to the database.

                                In our case, on the Starfire, the application perfomed much better when using 5-10 threads than using 20 or 30 of them. (I've read about other persons complaining about how Java 1.4.x scales on multi cpu machines). Note that we used Java 1.4.x here. I don't know about how Java 1.5 or Java 1.6 behave on multi cpu machines.

                                When we changed the environment to a normal Compaq D180 server (the pizza box) with only 2 Intel Xeon, 1GB RAM having better clock frequency, we got much better performance. I can't remember exactly how much better, but I know we are talking about several 100 percents. We had Java on one server and Oracle on antoher server (also a pizza box, but 4GB RAM). These machines used Windows 2000 Server Std. Edition.

                                Then we tried to seutp Oracle on a 8 Xeon cpu Dell server 8GB RAM, with Windows 2000 Server Ent. Ed (you had to have ent. ed. to be able to use all processors and all memory). This machine had lower processor speed than the piza box. This configuration gave us worse performance compared to the 2 pizza boxes.

                                On all machines we were in contact with Oracle about how to setup the Oracle to get the best performance. One thing Otracle learned me, but this was version 8.1.6, was to setup the chunk size of the disks. When you raid your disks, you shall have as specific chunk size. At least on windows. You shall also have a special raid configuration. From the beginning we had raid 10, but as I can remember you shall use something else.

                                Best insert performance:
                                1. import of flat file
                                2. creating a stored procedure wher you sent a kind of comma separated string.
                                3. Java - Oracle simple sql prepared statements
                                4. Java - using simple statements.
                                ...

                                What about the network between database and java. Are they running on separate machines. If not you will have a slower system.

                                Finally, Yes, the OCI drivers are much faster. Ther are communicating directly with the Oracle DB. The non OCI drivers are normally working with the PL/SQL in Oracle - an extra layer on top of the database.

                                Regards
                                Oskar


                                PS...
                                From Java, rember to turn off auto commit. Ds.

                                • 13. Re: 10,000 Recorc Per Second (In EJB 3.0)
                                  paata lominadze Master

                                  great thx for your posts, they are so useful for me,

                                  you are right snau2005, oci driver is more faster then thin driver but not very good for my issue,
                                  ok thx for your code about stored procedures, i will try it, if i could not fing any other way,

                                  i don't want stoberd procedures and bech inserts becouse in this way i won't have transaction management on my hand,

                                  and at last i tried to use Thread on server side, i created 10 threads and each of them tried to persist 1000 object into database, and i tried to flush after 50 records,

                                  i wrote my thread class into server side and use it into stateless session bean, and set entittymanager class into Thread class and into run method i tryed to persist

                                  public class ThreadMy extends Thread {
                                   EntityManager entityManager;
                                   List<UsageObject> usagesObjs;
                                   public ThreadMy(EntityManager entityManager,
                                   List<UsageObject> usagesObjs) {
                                   this.usagesObjs = usagesObjs;
                                   this.entityManager = entityManager;
                                   this.setPriority(Thread.MAX_PRIORITY);
                                   this.start();
                                   }
                                  
                                   public void run() {
                                   try {
                                   int count = 0;
                                   for (int i = 0; i < usagesObjs.size(); i++) {
                                   UsageObject usageObject = usagesObjs.get(i);
                                   entityManager.persist(usageObject);
                                   if (i / 50 >= 0) {
                                   entityManager.flush();
                                   count = 0;
                                   }
                                   count++;
                                   }
                                  
                                   } catch (Exception e) {
                                   e.printStackTrace();
                                   }
                                   }
                                  }
                                  


                                  and into my stateless sessionbean i have created 10 Threads

                                  but i got an error like this

                                  18:11:35,684 ERROR [STDERR] javax.persistence.TransactionRequiredException: EntityManager must be access within a transaction
                                  18:11:35,685 ERROR [STDERR] at org.jboss.ejb3.entity.ManagedEntityManagerFactory.verifyInTx(ManagedEntityManagerFactory.java:149)
                                  18:11:35,685 ERROR [STDERR] at org.jboss.ejb3.entity.TransactionScopedEntityManager.persist(TransactionScopedEntityManager.java:174)
                                  18:11:35,685 ERROR [STDERR] at com.magti.client.test.ThreadMy.run(ThreadMy.java:81)
                                  
                                  


                                  entityManager.persist(usageObject); // this is ThreadMy.java:81

                                  how i can fix it ?
                                  can anybody tell me ?


                                  ok and another thing, thank you also for your post oskar.carlstedt

                                  i am not an administrator of the network and systems into my company, i don't know servers very well, am a programmer.


                                  you thing it is more useful to use windows servers instead of solaris servers ?
                                  i thik if operation system is well configured, i think solaris is enough for my program and for this task.

                                  it is not right way that i requst my company to bye this servers which you tell me (Compac D180 or Dell Server) for my program,

                                  and what you say about best insert performance, i 100 % agree with you,
                                  i want good design and high performance together :) ;), maybe it is impossible, but i want try all ways, after all maybe is use comma separated files and stored procedures,


                                  PS ..
                                  of course i turn off auto commit into my java program,



                                  Regards,
                                  Paata.






                                  • 14. Re: 10,000 Recorc Per Second (In EJB 3.0)
                                    Oskar Carlstedt Novice

                                    Hi again!

                                    What I actually mean is that normally it is better to have a few really fast computers instead of a single powerful machine. And I understand your situation, you have this machine and you wan't to use it.

                                    Do you know if there are other IO-intensive tasks running on that machine at the same time? Normally you just don't buy such a server to insert 10000 rows/sec in an oracle database. Are you "alone" on your server?

                                    //Oskar

                                    1 2 3 Previous Next