This is a brainstorming document related to possible approaches to [WFLY-3510] support a form of modular XML configuration file - JBoss Issue Tracker and related issues like [WFLY-3509] git backend for loading/storing the configuration XML for wildfly - JBoss Issue Tracker. At this point it's far from a design; just a brain dump.

Requirements

(A proper requirements analysis should be the first step in really attacking this problem. I'm going to write down a lot of implementation-related thoughts in this document just to get them out of my head, but really we need a proper requirements analysis.)

Requirement -- ability to compose a configuration from separate filesystem elements, such that configuration bits can be added/removed/updated without requiring a change to an overall configuration document.

Note that this requirement rules out xi:include as a solution, as xi:include requires the addition of a tag in the including document.

Requirement -- ability for the server to detect changes and respond to changes in the configuration made via direct change to the persistent configuration; i.e. not via the native, http or JMX management APIs.

The exact form of any required response needs clarification; e.g. simply putting the server in reload-required state theoretically could be sufficient. My assumption at this point is we want something more than that though.

Anti-requirement -- any need to map individual resources in the runtime management model to random filesystem locations. For example, xi:include based solutions or piecing together an overall xml document from various xml snippets on the filesystem.

The fact that this is an acceptable anti-requirement is one of the key things that needs to be clarified.

Why is it an anti-requirement? To an extent this is a scope of work problem, as we have limited resources to devote to this overall area. Supporting this brings with it:

• Requirement to store the mapping data in the management model (not necessarily in the Resource tree, but somewhere)
• Likely requirement to expose that data via the management API
• Requirement to support passing in that data when adding new resources via the management API. We can't assume that the only time this gets used is at boot when a document is parsed. If someone uses the CLI to add a new datasource resource, they would need to have the ability to state where that resource should be persisted.
  • This means some sort of new global concept that can apply to any "add" operation
  • This concept would have to be handled by the core management layer; we can't expect subsystem authors to do anything about it
• Likely requirement to support changing this data post-add
• Requirement to ensure that the parsing and marshalling aspects of this are completely centrally handled with no impact on the parsers/marshallers developed by subsystem authors. Since subsystem provided parsers and marshallers would be responsible for parsing and marshalling the actual details of any configuration snippets, this would be a serious challenge.

Possible Approach

Persistence Format

Support storing a configuration in a filesystem tree that maps to the current Resource tree. The root of the tree maps to a directory on the filesystem. The model associated with that resource is represented by a single text file in that directory that contains a text representation of the DMR model, only including persistent attributes. Format of the file could be any format supported by the jboss-dmr library, currently the native DMR format and JSON. The file extension of the file would indicate what format is used. Child resources are represented by child directories, with a level for the PathElement key and another level for the value.

If someone wanted WildFly to support some other persistence format (say yml) if they provided the needed parsing/marshalling logic to jboss-dmr it would be pretty straightforward to add that.

Indication of desired format

Use the extension of the filename passed to --server-config, --domain-config, --host-config to control what kind of persistence is used by the process. So, standalone.xml indicates that the current xml persistence should be used. Something like standalone.dir indicates the filesystem tree approach is desired. Or, the dir itself could be named standalone.[dmr|json] to provide the parser with knowlege of the expected type of content.

(Question: would we want to support mixed formats? Say standalone.dmr is the main format, but at some sublevel torquebox adds in foo.yml for the YAML-based stuff they insert. Supporting this brings in a lot of the reasons why composing xml snippets is an anti-requirement, so I say no, we don't support this.)

Booting

Read the filesystem and create a Resource tree to match. Then use something like the "describe" behavior we have for launching servers to convert from a Resource tree to a set of management operations needed to create a real resource tree and to trigger execution of runtime steps.

At some point we may change how our boot works to eliminate the conversion of the persistent format to operations to execute, but that should be out-of-scope for this work.

Detect Post-Boot Changes

TBD. The git-based mechanism is a good possibility -- don't support direct changes to the local filesystem, but rather do git pulls. Another possibility is a scanner with a marker file system a la what we do with exploded deployments. Use a marker file to lock out the scanner until a set of changes is complete.

An interesting thing with a git-based approach is it allows some configurable ability to deal with local changes made to the model via the management API. For example, a policy option could be to discard local changes and reset to the remote configuration. Another option would be to attempt some sort of rebase/merge, failing if there is a conflict.

Respond to Changes

If a change is made, the effective change needs to be determined and then management ops should be executed to bring the running state into compliance with the configuration.

Roughly,

1. Create a resource tree to match the current persistent config state
2. Compare to the current config tree and generate a list of operations to bring the state into compliance with the tree
   1. Determine what resources need to be added
      1. For core resources, we'll require that there is an "add" op whose parameters match the persistent attributes
      2. For subsystem resources, check if there is an "add" op whose parameters match the persistent attributes
         1. If yes, use it
         2. If no, use the "describe" handler, passing in the relevant part of the resource tree from 1. to determine what operations must be invoked to add the resource
   2. Determine what resources need to be removed
      1. Formalize into a hard requirement the convention that any persistent resource must be removable via a no-param op named "remove"
   3. Determine what resources need to have individual attributes changes
      1. If the attribute is writable, use the write-attribute operation
      2. Otherwise, use a remove op for the entire resource and then add the resource per the "add" case above

Step 2 above would perhaps need to be executed as an operation, in order to ensure the model controller lock is held.

Handling Failure

TBD. This is the problem with allowing direct change to the config vs persisting a change that was made via the management API and thus validated -- if the change is invalid, it's already persisted.

The git-based approach is interesting for this, as it offers the possibility of simply reverting the local filesystem back to what it was before any invalid change was pulled from the remote git repo.

Issues

Upgrades

Our current xml-based system relies on the ability for parsers for legacy namespaces to create management ops that will comply with the current management API. How would we handle this without the xml parsers?