Tools for multi-environment consistency - jboss xml formatted input to dmr commands

[vandamo]

Hi,

I've been searching the various available resources regarding an approach for environment management that would be suitable to my current company without success,

so am now turning to the JBoss community for an input.

I apologize in advance if this question has been asked before, would be happy to be pointed to it and its resolution if it's the case.

JBoss version: EAP 6.2.0.GA

java.version: 1.7.0_25

OS: Red Hat Enterprise Linux Server release 6.5 (Santiago)

Goal: script/automate the setup of JBoss environments.

Preferred input: XML, following JBoss schema format.

Constraint: this should not limit us to:

1. full stop

2. push xml and all additional elements in the environment

3. restart

as at least for production environment, we should be able to consider no downtime for deployment as long as some key elements have not been altered (most obvious case would be database).

What I was looking for:

1. Preparing the configuration for an environment via XML, validating it against the schemas made available in JBoss.

2. Parsing the XML to push the configuration block by block or something similar, using the dmr api, with minimal overhead for mapping xml to dmr commands.

The logic behind this approach is that we would have in theory as input the "result" of the deployment also, meaning the xml configuration provided should match the xml configuration generated by the commands.

As JBoss uses the XML configuration as it's main input (at startup), it implies the parsing of said configuration to figure out the general setup.

Reviewing with dmr-echo the expected input for (for example) a jdbc-driver creation, it is apparent that there is no such obvious mapping between xml and mdr json-like input.

So comes the question, why?

Is the approach we are considering flawed for some reason?

Would an input (XML or other) not related to the expected output be a more viable option and if so what is the logic behind such conclusion?

If the approach is sound and considered by others, is such an implementation available somewhere (free or not)?

Thanks in advance for your feedback.

Message was edited by: Olivier Van Dam Added JBoss version info and OS.

[jamezp]

I'd advise you to use CLI batch files. You just need to create a file with CLI commands and execute it via the jboss-cli.sh/.bat. See CLI Batch Mode and CLI Recipes - JBoss AS 7.2 - Project Documentation Editor for some details. There is also some if-else control flow and try-catch-finally control flow you can use.

The DMR is written out as XML. You might be seeing defaults that DMR adds which the XML just doesn't write out. Not sure there, I'm just guessing.

--

James R. Perkins

[vandamo]

Hi James,

Thanks for your input.

I had already looked into CLI commands prior to posting the question, what I don't understand is that since the configuration is kept in xml format, and a clearly defined one at that, that the "internal tools" jboss uses to parse that configuration and create the running instances is not accessible in one way or another.

I would have liked the option of creating a domain.xml and multiple host.xml for each of my environments, use that as a starting point for the original deployment (without having to go through much additional processing as it would be the first setup),

then maintain those xml and apply the modifications related to our needs, for each of our environment, and be able to use that as an input for "online" configuration change.

As it is, I'm now considering an xml configuration for applying those changes, that would closely match the operations cli input [path]:[command]([parameters]).

Then use jython to parse the aforementioned xml and build up the cli commands.

I'm hoping I can easily build around that to add some simple if [exists] then [delete]; [add] logic so that the scripting stays simple, and the configuration is readable and easily adapted for each of our environments and applications.

In that context your input about try-catch-finally is very interesting, my investigations had only uncovered the if-else control flow.

Thanks again for your help.

This leaves one element I'm not thrilled about, being that the xml script input structure will not be at all comparable to the resulting xml configuration in jboss.

I'll leave this discussion opened, in case someone would wish to provide input on either the original question, or the approach we are currently considering.

[jamezp]

Ideally there would be no configuration file at all. Clearly that can't happen.

While the subsystems and the controller can parse the XML, it's not really meant to be any kind of public API. An XML configuration is currently used, but there is no real reason it's XML. It could be plain DMR, JSON, etc. The XML is overwritten if anything changes. Any comments you add will be lost. The configuration files are not really meant to be maintained by hand so to speak.

There are additional benefits to using the management options, like CLI or the web console, too. For example if you wanted to turn on debug logging. If you edit the XML you first have to stop the server, edit the XML then start the server. If you use CLI or the web console you just need to add the logger and/or change the log level and it's done. No need to restart anything and the XML will be updated itself.

--

James R. Perkins

[vandamo]

I understand the benefits of CLI or Web interface, but was hoping to use the existing schemas defined by JBoss for the configuration format to be kept and versioned.

As in my current environment, after migration of our applications on JBoss we'll end up with about 20 server nodes per environment, and we have a total of 7 environments, I need to maintain the configuration centrally so that any modification in the dev environment can be easily replicated to the other environments.

An additional consideration being that I don't have access to the production environment, so I need to provide scripts to perform the task there.

It feels more maintainable to have the configuration for each environment in an xml (or other ini, json, ...) format and a "simple" parsing script that would convert it to cli commands.

This would mean that the configuration can be separated from the if [exists] remove, [add] structures, and hopefully the xml would be clear enough (and easily validated) so that anyone can modify, and even add full sections related to a new server group or other configuration.

Thanks for your reply

[jamezp]

With that many nodes have you considered looking at domain mode? I'm no expert on it, but it gives you a way to centrally manage multiple servers and hosts. This is for WildFly, but it should be the same for EAP Domain Setup - WildFly 8 - Project Documentation Editor. Semi related you can boot servers into admin-only mode too where you just make configuration changes, via CLI, the web console, JMX, etc.

--

James R. Perkins

[vandamo]

Actually I don't have a say-so on this, the company has indeed gone for that option (being domain configuration, available with EAP 6.2.0.GA,

but have imposed a separate domain by environment.

So I'll have indeed a partial configuration centralisation.

Thanks again for your input.

[vandamo]

Ok, followup on the current status of my work (to share with whomever is interested - and have a reference for questions later),

I have a first draft of jython script to process an xml file containing the details of the cli commands to be run:

In attachment the jython first draft (not much in the sense of exception handling, some functions oversimplified due to original, logic, ...), as well as a sample of the xml configuration.

What I'd like now is to figure out the syntax for upload of the application...

I'd rather continue with the /<path>:<operation>(<list of attributes>), but haven't found the syntax to use.

This would allow for a more systematic approach.

Tests and searches show that I could maybe use:

/deployment=other.war:add(content=?????

But I'm not sure what options and parameters to use when using this through the cli.cmd method from jython.

Input welcome, as it's the best way for me to learn.

[vandamo]

Found this recently: [WFLY-693] CLI "revert to snapshot" command - JBoss Issue Tracker

Which if it was to be implemented one day, could possibly translate to a rough version of what I was originally looking for