8 Replies Latest reply on Apr 8, 2009 4:53 PM by jnusaira

    Batch Processing with Seam?

    jnusaira

      I was basically curious how many of you accomplished it and if its possible to do it with jBPM?


      Basically ... i have to upload to the server TONS of files of various sizes. These files then contain blocks of data that need to be handled by another service.


      And in fact ... a couple times you haev to handle ALL of a certain file set before you can start another file set.


      But most importantly i want to run everything asynchronously with many services processing files X at a time.


      I've been looking at Spring Batch, but that seems to have some workflow in it as well ...


      anyway ... any ideas?

        • 1. Re: Batch Processing with Seam?

          is it advisable to use Seam and Spring Batch Processing? have you found it to be a nightmare or is it a good combination?
          are there any other ways to face huge batch processing with seam?
          thank you!


          • 2. Re: Batch Processing with Seam?
            jnusaira

            It was a good and bad experience i have to say. (although the bad part was mainly i think cause of our environment)


            So the good part. Using spring batch was pretty easy once oyu figure out HOW you want to configure it and HOW to use it. I basically had my spring readers and writers call out to my Seam components. So injecting both ways was VERY simple.


            The bad news.


            So this wasn't a traditional batch job. In that you had to trigger it from a web site. So i tried to use the @Async via quartz to trigger it; which to use you have to be a Java Bean Seam component and NOT an EJB3 Seam component. (the ejb3 one never seemed to actually trigger it asyncrhonously)


            Soo ... i launched it. And would work fine until it reached one of our WebServices we have to use. And for some reason i would get a million classpath issues. So still not sure why that was. Ended up resolving it by just having the item sent to an MDB and have the MDB trigger the batch process and worked fine after that.


            Anyway ... hope that helped. ... so yeah i'd recommend it if batch processing is what you needed. The main reason i wanted to use spring batch is basically we have to parse tons of files and process them. So i wanted to have many threads going for process A. But then i CANT start process B till ALL of A is complete. And it worked out great for that

            • 3. Re: Batch Processing with Seam?

              So, if you use seam-gen for a start the best choice is the war and not the ear one. I mean, to have POJOS and not EJBs in terms of async. Otherwise, if you have an ear architecture you need to have as many pojos as needed as for async issues.
              thank you very much Joseph!


              Seam guys, is there a plan to create a red hat-seam product such as spring batch for this issues?
              I love seam and I´m been using it for almost two years. And according to what have been said, I guess I´ll have mix Spring Batch with seam apps if massive batch processing is required.

              • 4. Re: Batch Processing with Seam?
                tariqahsan

                Hi Joseph,


                I am very much interested know how and what are the pieces of Seam and Spring you have integrated in your mentioned batch process. We are trying to re-design an age old unix kornshell based batch processing application to fully Java based one. We are already using JBoss, jBPM, Seam Framework API, Oracle 10g, EJB 3.0 for our web application.
                If you could share your insight and experience that would greatly help.


                Thanks

                • 5. Re: Batch Processing with Seam?
                  jnusaira

                  The integration went pretty easily.


                  I am using Spring batch to drive most of the processing and moving along from step to step.


                  So when i use seam, basically i inject Seam into the Reader/Writers. That way i am using my main business logic components there using Seams seam:intance spring property.


                  Initially i used the @Asynchrnous Seam annotation on a javabean to call the spring JobLauncher which then calls reader/writers which call the seam components. These then called out to some webservices. This actually caused me some problems with classpath errors (For the webservices only). So instead i changed to have an MDB call the JobLauncher instead.


                  It's definitely feasible and very easy.

                  • 6. Re: Batch Processing with Seam?
                    tariqahsan

                    Joseph,


                    Thanks for sharing your experience integrating Spring Batch with Seam. We would certainly would look into this trying out ourselve for converting our existing batch process application.


                    Again much appreciated for your response.


                    Tariq

                    • 7. Re: Batch Processing with Seam?
                      tariqahsan

                      Joseph,


                      One other question about setting up 'step' etc. Spring Batch tags. Are you using the jbpm.org.jpdl and/or jbpm.org.xsd for creating your processdefintion.xml? If so how are you doing so using the Spring Batch tag's in it? I think it is a trivial question to ask. Could you share some example of this type of xml file if it is not too much to ask?


                      Thanks again.

                      • 8. Re: Batch Processing with Seam?
                        jnusaira


                        jBPM isnt a great way to do batch processing. It can be done by spawning a bunch of nodes and you can tell them to wait. The only problem with it is that it is going to create quite a bit of database entries if you have lots of data you are running through.


                        That was our initial strategy in fact. But all you are creating is a linear JBPM flow and thats really a waste.


                        The way i look at jBPM vs  spring batch is this. jBPM is GREAT at taking ONE item and running through a bunch of complex servies, decision nodes, tasks, etc. It handles the heck out of one thing in a complex manner.


                        Spring batch takes millions of items through the same linear process great. The process has to be linear though because you really cant say well if they dont all execute do this etc. We do throw exceptions at times to stop the process but the two arent really the same or compatible.


                        Now you COULD if each item has a complex path have your writer feed into a jBPM pathway. We do use jBPM here but for different processing rules