Programmatically creating datasource using dmr hangs

[yangju]

Here is my code:

public void createDatasourceProgrammatically() throws Exception{

        ModelNode op = new ModelNode();

        ModelControllerClient client = null;

        try {

            op.get(ClientConstants.OP).set(ClientConstants.ADD);

            op.get(ClientConstants.OP_ADDR).add("subsystem", "datasources");

            op.get(ClientConstants.OP_ADDR).add("data-source", "java:jboss/datasources/ds1");

            op.get("jndi-name").set("java:jboss/datasources/ds1");

            op.get("connection-url").set("jdbc:mysql://mydomain:3306/ds1?zeroDateTimeBehavior=convertToNull&useUnicode=true&connectionCollation=utf8_general_ci&characterSetResults=utf8&characterEncoding=utf8&useInformationSchema=true&noAccessToProcedureBodies=true&autoReconnect=true&autoReconnectForPools=true");

            op.get("driver-class").set("com.mysql.jdbc.jdbc2.optional.MysqlXADataSource");

            op.get("driver-name").set("com.mysql");

            op.get("user-name").set("aaa");

            op.get("password").set("bbb");

            op.get("pool-name").set("ds1");

            client = createClient(InetAddress.getByName("127.0.0.1"), 9990, "admin","admin","ManagementRealm");

            ModelNode returnVal = client.execute(op);

        }

        catch(Exception e) {

           log.error("faile to execute", e);

        }

        finally{

          if(client!=null){

              client.close();

          }

        }

    }

private ModelControllerClient createClient(InetAddress host, int port,

            String username, String password, String securityRealmName) {

        final CallbackHandler callbackHandler = new CallbackHandler() {

            public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {

                for (Callback current : callbacks) {

                    if (current instanceof NameCallback) {

                        NameCallback ncb = (NameCallback) current;

                        ncb.setName(username);

                    } else if (current instanceof PasswordCallback) {

                        PasswordCallback pcb = (PasswordCallback) current;

                        pcb.setPassword(password.toCharArray());

                    } else if (current instanceof RealmCallback) {

                        RealmCallback rcb = (RealmCallback) current;

                        rcb.setText(rcb.getDefaultText());

                    } else {

                        throw new UnsupportedCallbackException(current);

                    }

                }

            }

        };

        return ModelControllerClient.Factory.create(host, port, callbackHandler);

}

When I ran above code in wildfly 11.0.0.Final server, the execution of client.execute(op) hangs forever.

I use the similar code but just to do some read operation, and it returns result immediately.

Maven dependencies:

<dependency>

   <groupId>org.jboss</groupId>

   <artifactId>jboss-dmr</artifactId>

   <version>1.4.1.Final</version>

  <scope>provided</scope>

</dependency>

<dependency>

   <groupId>org.wildfly.core</groupId>

   <artifactId>wildfly-controller-client</artifactId>

   <version>3.0.8.Final</version>

   <scope>provided</scope>

</dependency>

Any help is appreciated.

[jamezp]

What line is it holding on? The callback is likely not needed since it looks like it's executing on the a local server.

--

James R. Perkins

[yangju]

Hi, James, thanks for replying.

It hangs on this line:

ModelNode returnVal = client.execute(op);

I think it is because this line is invoked while the wildfly server is still being started and therefore causes a deadlock in the deployment.

My goal is that after the server is started, during a deployment of the application, I have this producer @produces or @startup application scoped service which adds datasources programmatically based on some external configurations. I don't know if there is better way to achieve this. How would you do this?

The standalone.xml has already had some entries in the <datasources> section for some fixed datasources. My goal is to add more datasources to this section at deployment time.

1) Is there way to know if the server is fully started and then before anything else is deployed, this datasource adding service or producer will be invoked?

2) The other thing is that I am not sure these two lines are correct if I want to add a new datasource to an existing <datasources> section:

op.get(ClientConstants.OP_ADDR).add("subsystem", "datasources");

op.get(ClientConstants.OP_ADDR).add("data-source", "java:jboss/datasources/ds1");

I could not find detailed doc on the dmr api(s).

3) As for the callback on localhost, my wildfly admin console has password enabled. So without using the callback, how can I pass the userId and password to the ModelControllerClient.Factory.create()?

Could you answer above three questions I have.

Thanks again.

Richard

[jamezp]

Hello Richard,

You may want to look at using an *-ds.xml. Here's an example. If it didn't freeze there would be a race between when the operation executes and when the resources which require the data source are processed. In fact the resources might even be processed before the operation is executed which would cause deployment failures.

yangju  wrote:

1) Is there way to know if the server is fully started and then before anything else is deployed, this datasource adding service or producer will be invoked?

The deployment process shouldn't start until the server is fully started. That said it might be too late in the process to be adding a data source like this.

yangju  wrote:

2) The other thing is that I am not sure these two lines are correct if I want to add a new datasource to an existing <datasources> section:

op.get(ClientConstants.OP_ADDR).add("subsystem", "datasources");

op.get(ClientConstants.OP_ADDR).add("data-source", "java:jboss/datasources/ds1");

I could not find detailed doc on the dmr api(s).

That looks correct. There is a utility you can use as well to help with DMR. Here's an example if the code converted to use the Operations helper

public void createDatasourceProgrammatically() throws Exception {
    try (ModelControllerClient client = ModelControllerClient.Factory.create(InetAddress.getLocalHost(), 9990)) {
        final ModelNode address = Operations.createAddress("subsystem", "datasources", "data-source", "ds1");
        final ModelNode op = Operations.createAddOperation(address);
        op.get("jndi-name").set("java:jboss/datasources/ds1");
        op.get("connection-url").set("jdbc:mysql://mydomain:3306/ds1?zeroDateTimeBehavior=convertToNull&useUnicode=true&connectionCollation=utf8_general_ci&characterSetResults=utf8&characterEncoding=utf8&useInformationSchema=true&noAccessToProcedureBodies=true&autoReconnect=true&autoReconnectForPools=true");
        op.get("driver-class").set("com.mysql.jdbc.jdbc2.optional.MysqlXADataSource");
        op.get("driver-name").set("com.mysql");
        op.get("user-name").set("aaa");
        op.get("password").set("bbb");
        op.get("pool-name").set("ds1");
        
        final ModelNode result = client.execute(op);
        if (!Operations.isSuccessfulOutcome(result)) {
            log.error(Operations.getFailureDescription(result).asString());
        }
    } catch (IOException e) {
        log.error("faile to execute", e);
    }
}

Again though you'll likely not want to do this from within the deployment.

yangju  wrote:

3) As for the callback on localhost, my wildfly admin console has password enabled. So without using the callback, how can I pass the userId and password to the ModelControllerClient.Factory.create()?

By default it will attempt to use local authentication. This can be disable so if you disable it you'd need to use the callback handler.

--

James R. Perkins

[yangju]

Thanks James for your suggestions. After I moved the call to programmatically add datasources out of my bean's @PostConstruct method, it does not hold up the deployment anymore and the dynamic datasource creation seems to be working. I am planning to write a startup servlet to do this datasource creations after the application is deployed but before anything is invoked from end users.

However, I still have a few datasources which are required by JPA's persistence.xml and I cannot create them like other datasources that I can programmatically create at run time. So I am thinking using your proposed  -ds.xml approach (currently all of my datasources are predefined in the standalone.xml ahead of deployment).

If I use -ds.xml files in deployments directory, will those -ds.xml files be deployed earlier than the .war or .ear files in the same directory? If so, I think JPA deployment will not have problem.

Please let me know if this approach is ok or is there a better approach?

Richard

[jamezp]

Yeah the -ds.xml files are processed early enough to were it will work with JPA.

--

James R. Perkins

[yangju]

But if I deploy datasource as -ds.xml, do I lose the stats and also can I still manage the datasources from jmx console or jboss-cli such as flush the connections, etc?

[jamezp]

Yes you would lose those capabilities.

--

James R. Perkins

[yangju]

James, another related question.

So since my application creates the datasoures programmatically, the standalone.xml is updated with the datasources (the changes are persisted).

This will create a situation that next time the application is started, I will have to replace those datasource entries and recreate, as the datasources configuration may have changed.

So my questions are:

1) is there way or settings in the dmr interface that the changes are only set in memory and not to be persisted to standalone.xml?

2) If 1) not not possible, what is the correct dmr method I can use to replace the existing datasources and override them programmatically?

Thanks.

Richard

[jamezp]

yangju  wrote:

1) is there way or settings in the dmr interface that the changes are only set in memory and not to be persisted to standalone.xml?

2) If 1) not not possible, what is the correct dmr method I can use to replace the existing datasources and override them programmatically?

Hi Richard,

The only way I could think of is to use the --read-only-server-config option when starting up the server. That however means every time you restart any changes made will be lost.

You can also check to see if the datasource already exists. Something like the following should work:

private static void configureDatasource() throws IOException {
    try (ModelControllerClient client = ModelControllerClient.Factory.create(InetAddress.getLocalHost(), 9990)) {
        if (!datasourceExists(client, "TestDS")) {
            final ModelNode address = Operations.createAddress("subsystem", "datasources", "data-source", "TestDS");
            final ModelNode op = Operations.createAddOperation(address);
            op.get("jndi-name").set("java:jboss/datasources/TestDS");
            op.get("connection-url").set("jdbc:h2:mem:test-2;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE");
            op.get("driver-name").set("h2");
            op.get("user-name").set("sa");
            op.get("password").set("sa");
            op.get("pool-name").set("test-ds-pool");
            executeForSuccess(client, op);
        }
    }


}


private static boolean datasourceExists(final ModelControllerClient client, final String name) throws IOException {
    final ModelNode address = Operations.createAddress("subsystem", "datasources");
    final ModelNode op = Operations.createOperation(ClientConstants.READ_CHILDREN_NAMES_OPERATION, address);
    op.get(ClientConstants.CHILD_TYPE).set("data-source");
    final ModelNode result = executeForSuccess(client, op);
    for (ModelNode dsName : Operations.readResult(result).asList()) {
        if (name.equals(dsName.asString())) {
            return true;
        }
    }
    return false;
}


private static ModelNode executeForSuccess(final ModelControllerClient client, final ModelNode op) throws IOException {
    final ModelNode result = client.execute(op);
    if (!Operations.isSuccessfulOutcome(result)) {
        throw new RuntimeException(Operations.getFailureDescription(result).asString());
    }
    return result;
}

--

James R. Perkins

[yangju]

James,

I think your proposed solution about "check if Datasource does not exist then add it" works.

Now I move on to the next part, I need to figure out a way to see if the deployment is completed. We deploy multiple war files inside the deployments folder.

I am thinking to use DMR to do something jboss-cli does:
/deployment=*:read-attribute(name=status) where the status or result is "OK"

Because I cannot hard code the war file name, I would need a wildcard such as "*.war", but it does not seem to work for me.

Could you provide a java example for DMR to say that:

Is it

All war files inside deployments are successfully deployed?

If not, my bean around creating datasources will sleep and then wake up to do above check again, until all files are deployed. Then the bean will start to do the datasource checking and creating.

Thanks.

Richard