Executing batch jobs (JSR 352) in a non HA WildFly (full-profile) cluster

[damzgaard]

Hi there,

Let me set the scene:

• I'm in the process of designing a solution that requires that we use batch jobs using the new JEE 7 batch API.
• We are planning to have more than one WildFly (full-proflile) server running at any time but not in high availability configuration - simple external load balancing HTTP will do fine here.
• We are planning to deploy the JAR containing the batch job configuration and the beans to support on all containers.
• We are planning to configure the change the job-repository to JDBC in a shared database as described here: http://www.mastertheboss.com/wildfly-8/configuring-wildfly-batch-job-repository
• We are planning to use Quartz Scheduler running in another container to control the execution using a rest-service running on any of the WildFly instances to control the execution.

In other words - clustered batch execution using WildFly.

Questions:

1. Is this a valid configuration?
2. Is it guaranteed that a job only execute in / on one container?
3. Can a job be controlled (started / re-started / canceled etc) from any container instance?

I have thought about creating a proof-of-concept setup with two WildFly containers using this configuration but since this involve multiple JVM / servers running in parallel I won't be able to say anything conclusive in case of a positive result which might just be due to poor luck.

Please let have your toughest on this subject.

Best regards

Christian

[jamezp]

I don't believe batch is currently supported in a cluster. I know there was at least nothing special done at this point. It has been discussed, but that's as far as it's gone at this point.

--
James R. Perkins

[damzgaard]

Hi James,

Thank for the information.

I have done some investigations on my own and I currently see two issues with the implementation as it is in 8.1:

1. Batch jobs are not loaded from the META-INF/batch-jobs directory at launch time nor at first access to the BatchRuntime.getJobOperator()
2. Batch job repository is not loaded from the database at launch time nor at the first access to the BatchRuntime.getJobOperator()

Both issues are somewhat related and makes it hard to monitor the system since it will report no batch job know after a restart of the JVM - i.e. how can one know that a batch job is failed if it is not listed at all after a JVM restart?

Am I getting things wrong here?

Best regards

Christian

[fcorneli]

Hi Christian,

I've stumbled upon the same 'issue'. According to the JSR352 specs for JobOperator.getJobNames:

"Returns a set of all job names known to the batch runtime."

The rationale behind this behavior (in JBeret) is that the batch runtime only knows about jobs once it has started one. However, via for example JobOperator.getJobInstances you can always query for job instances/executions, even while getJobNames still gives you an empty set. So within your monitoring you need to know about the job names in advance.

Kind Regards,
Frank.

[damzgaard]

Hi Frank,

Thank you for your input - I'll look into the usage of getJobInstances instead.

I reoccn that the reason for the BatchRuntime not to discover the batch job files are that there is no Java API for listing content of the classpath without parsing the .jar file using the .zip file API.

Using the META-INF/INDEX.LIST file could be an option to detect the batch files automatically, but unfortunately the Maven archiver plug-in in broken in this area http://jira.codehaus.org/browse/MASSEMBLY-657 and not likely to be fixed any time soon since it was reported over an year ago.

Best regards

Christian