WildFly 9.0.2 MBean Start Method Timeout/Deadlock
I am currently in the process of migrating a suite of enterprise applications from JBoss AS 4.2.2 to WildFly 9.0.2.
I have encountered an issue deploying one application with WildFly 9.0.2 in particular, and I am not sure how to address it.
I will provide an overview of the application and the error information I have been able to obtain.
Application Overview:
The application, which I will refer to as DataSourceManager, is a small EAR application that is responsible for dynamically loading and managing data sources. Basically, it allows data sources to be defined without the use of -ds.xml files in JBoss 4.2.2 or direct manual configuration of the standalone/domain.xml file in WildFly 9.0.2.
When DataSourceManager is deployed, it dynamically loads and configures data sources [the metadata/configuration data for the data sources are dynamically provided by an external source]. DataSourceManager is deployed before other applications in the suite so that the data sources the other applications need are already configured when they are deployed.
Implementation:
From a code standpoint, the DataSourceManager consists of two classes--an MBean interface and its implementation.
The interface:
public interface DataSourceManagerMBean {
//life cycle method
void start() throws Exception;
...
}
The implementation:
public class DataSourceManager {
public void start() throws Exception {
loadDataSources();
}
...
private void loadDataSources() throws Exception {
//DataSourceConfig encapsulates the data source configuration data. Defined in external library.
final Collection<DataSourceConfig> dataSourceConfigs = ...;//Provided by external source
for(DataSourceConfig config : configs) {
loadDataSource(config);
}
}
private void loadDataSource(DataSourceConfig config) throws Exception {
//Load data source using JBoss/WildFly MBean operations. These operations will differ between JBoss 4.2.2 and WildFly 9.0.2.
final MBeanServer mBeanServer = ...;
final ObjectName mBeanName = ...;
final String operationName = ...;
//Some parameter values are provided by the DataSourceConfig method parameter
final Object[] parameters = ...;
final String[] signature = ...;
mBeanServer.invoke(mBeanName, operationName, parameters, signature);
}
}
EAR Structure:
The EAR for the DataSourceManager is structured as follows:
datasourcemanager.ear
| - lib
| - datasourcemanager.jar //contains DataSourceManagerMBean and DataSourceManager
| - ... //other library JARs
| - META-INF
| - application.xml //probably not necessary, as no Java EE modules are defined, but otherwise harmless as far as I can tell
| - jboss-app.xml //defines a module for the SAR
| - jboss-deployment-structure.xml //Defines dependencies on JDBC driver modules. This deployment does not depend on any other deployments.
| - MANIFEST.MF //nothing of note here
| - datasourcemanager.sar
| - META-INF
| - jboss-service.xml // defines the single MBean for the application
| - MANIFEST.MF //nothing of note here
Deployment:
The application is deployed to a standalone WildFly server using the file system deployment scanner. I understand that using the appropriate management APIs is the preferred approach; however, this is currently not an option. As a side note, I am aware that WildFly has a CLI that can be used to accomplish what the DataSourceManager application does; however, I would prefer not to use this, and I am likely not in a position to use the CLI [due to external factors and forces that are beyond my control].
The Issue:
Getting this part of the DataSourceManager to work in WildFly 9.0.2 required some changes, due to differences in which MBean operations are used to configure data sources between WildFly 9.0.2 and JBoss 4.2.2 [WildFly 9.0.2 also has an additional step for loading JDBC drivers]. While migrating the code, I temporarily removed the call to loadDataSources() in the start() method to isolate any issues with the method from other deployment issues and exposed it for testing via JConsole. The loadDataSources() method works in WildFly 9.0.2 as well as, if not better than, it did JBoss 4.2.2; however, what does not work in WildFly 9.0.2 that did previously is calling the method from the start() method.
Once I put the loadDataSources() back into the start() method, the deployment process of the datasourcemanager.ear hangs for about five minutes before it times out. While I have not extensively measured execution times for the loadDataSources() method, the method typically executes in 1-2 seconds, which is nowhere near as long as the deployment process hangs.
The relevant information I was able to get from the command line [the server.log has an error entry for this in abbreviated form] is in the "datasource_deploy_error_filtered.txt" attachment.
If anyone has any ideas about how to address this issue or could tell me if what I am doing is even possible in WildFly 9.0.2, this would be very much appreciated.
