[marklittle]

The document mixes some requirements with implementation decisions, so it's hard for me to see what you might mean by "modernisation". Is this just removing the reliance on global configuration state?

[dmlloyd]

Mark Little wrote:

The document mixes some requirements with implementation decisions, so it's hard for me to see what you might mean by "modernisation". Is this just removing the reliance on global configuration state?

More generally, just removing reliance on global state, period.

[marklittle]

OK, so I agree with you that the classloader approach is sub-optimal, but I also agree with Paul that it is the right first step to at least determine if WF can benefit from using ArjunaCore for the MSC. Assuming this proves to be the case, then any change to ArjunaCore to remove reliance on global state should be opaque to the MSC anyway, since you will be utilising it through interfaces which will not be changed by any subsequent modifications to ArjunaCore.

[tomjenkinson]

Could implementing the classloader approach be achieved by running two separate subsystems in WildFly, as I understand it that would leverage the classloader isolation provided by jboss modules?

It might mean binding something new into JNDI in order that MSC can get a reference to an AtomicAction "factory"/similar but that should be possible.

[dmlloyd]

Mark Little wrote:

OK, so I agree with you that the classloader approach is sub-optimal, but I also agree with Paul that it is the right first step to at least determine if WF can benefit from using ArjunaCore for the MSC. Assuming this proves to be the case, then any change to ArjunaCore to remove reliance on global state should be opaque to the MSC anyway, since you will be utilising it through interfaces which will not be changed by any subsequent modifications to ArjunaCore.

Yeah, that's why I said "beyond the scope of our requirements" and other disclaimer-ish language.  My instinct is always to look beyond the immediate use case, even though that's a hard thing to justify in concrete terms I guess.

[dmlloyd]

Tom Jenkinson wrote:

Could implementing the classloader approach be achieved by running two separate subsystems in WildFly, as I understand it that would leverage the classloader isolation provided by jboss modules?

It might mean binding something new into JNDI in order that MSC can get a reference to an AtomicAction "factory"/similar but that should be possible.

Nah the class loading is orthogonal to management subsystems.  The ArjunaCore bits we would use for MSC and management would actually not be directly managed by the management system - at least not overtly.  The easiest way to do this is to just introduce another module (i.e. another copy of the bits) which contains just the parts we want to use for this.

[dmlloyd]

David Lloyd wrote:

Tom Jenkinson wrote:

Could implementing the classloader approach be achieved by running two separate subsystems in WildFly, as I understand it that would leverage the classloader isolation provided by jboss modules?

It might mean binding something new into JNDI in order that MSC can get a reference to an AtomicAction "factory"/similar but that should be possible.

Nah the class loading is orthogonal to management subsystems.  The ArjunaCore bits we would use for MSC and management would actually not be directly managed by the management system - at least not overtly.  The easiest way to do this is to just introduce another module (i.e. another copy of the bits) which contains just the parts we want to use for this.

Also we won't even have JNDI at this stage - this is the "back door" of the app server so there's no thread pools, no JNDI, no EE, no web server, no sockets, etc.

[dmlloyd]

Paul Robinson wrote:

David,

My current thoughts are that we can get classloading to do most of the heavy-lifting here and prevent too-much disruption of the Narayana code-base. Are you able to take a look at the "Overview of the solution" section and let me know if you think I am heading in the right direction? I suspect the main problem would be that we are simplifying the development work at the cost of performance and memory footprint due to there being two TMs running.

Thanks,

Paul.

One thing I want to point out here is that a key use case of the new streamlined management system is making embedded use of the server infrastructure easier (and by easier I mean "humanly possible").  So assuming we prove out this idea, the class loading solution can never be more than temporary (else I'll have to hack up some kind of infrastructure to replicate modules, which could be quite ugly - and if effort is spent I'd rather it be done to improve a thing rather than add odd one-off hacks).

[tomjenkinson]

David Lloyd wrote:

Tom Jenkinson wrote:

Could implementing the classloader approach be achieved by running two separate subsystems in WildFly, as I understand it that would leverage the classloader isolation provided by jboss modules?

It might mean binding something new into JNDI in order that MSC can get a reference to an AtomicAction "factory"/similar but that should be possible.

Nah the class loading is orthogonal to management subsystems.  The ArjunaCore bits we would use for MSC and management would actually not be directly managed by the management system - at least not overtly.  The easiest way to do this is to just introduce another module (i.e. another copy of the bits) which contains just the parts we want to use for this.

Hi David,

Sorry, I think I may have used the wrong terminology here, what it turns out I mean is I wonder if we can prototype (achieve?) the classloader separation using two separate "JBoss Modules" in WildFly. As you have identified I would expect the same jar to be present in the two separate "modules" where the "msc" depends on an "arjunacore:main" module and "transactions subsystem" depends on the existing "jts:main" module.

Is "JBoss Modules" available at the stage that we would need it, I can appreciate that subsystems are not.

JNDI would not actually be a requirement. We can have something like:

AtomicActionFactory aaf = AtomicActionFactory.getAtomicActionFactory(String configurationType);

Thanks,

Tom

[paul.robinson]

David Lloyd wrote:

Paul Robinson wrote:

David,

My current thoughts are that we can get classloading to do most of the heavy-lifting here and prevent too-much disruption of the Narayana code-base. Are you able to take a look at the "Overview of the solution" section and let me know if you think I am heading in the right direction? I suspect the main problem would be that we are simplifying the development work at the cost of performance and memory footprint due to there being two TMs running.

Thanks,

Paul.

One thing I want to point out here is that a key use case of the new streamlined management system is making embedded use of the server infrastructure easier (and by easier I mean "humanly possible").  So assuming we prove out this idea, the class loading solution can never be more than temporary (else I'll have to hack up some kind of infrastructure to replicate modules, which could be quite ugly - and if effort is spent I'd rather it be done to improve a thing rather than add odd one-off hacks).

I probably don't understand enough about how the "embeded use of the server infrastructure" would work, but here's my understanding:

There will be a separate project (to WildFly) containing the kernel of the AS. This will contain just enough to manage the lifecycle of installed Subsystems, provide management functionality and some other low-level things. There will then be other projects that build on this infrastructure; one such project being WildFly. In the WildFly case, a number of subsystems would be provided that when installed in the "kernel" would provide the Application Server as we currently know it. Any java libraries required by this kernel would be used just by it and not be visible in the class loaders used by the installed Subsystems.

So, assuming what I say above is mostly correct, I would see MSC in the "kernal" having a dependency on ArjunaCore with a TransactionManager and Recovery Manager initialized and running in the kernel's class loader. I would then imagine that subsystems installed in the kernel, would use JBoss Modules to manage their dependencies. In which case, any subsystem (or application) could depend on the current org.jboss.jts module if transactions which would run in a different class loader and thus be isolated from the kernel's instance of ArjunaCore.

[paul.robinson]

All,

I've knocked up a quick prototype [1] that uses JBoss Modules to provide classloader isolation. I'm sure we all (especially David) already know what this would look like, but I thought it would be a good idea to create something so that I can make sure I understand it properly and so that we have something tangible to discuss.

In the prototype, I've created a new JBossModule called "org.jboss.narayana.arjunacore". This module has its own jbossts-properties.xml (located at /modules/system/layers/base/org/jboss/narayana/arjunacore/main/config/jbossts-properties.xml) for configuring this instance of ArjunaCore. In the property file I have changed the object store location to be in "./dummymsc-object-store"; this allows us to see that two separate configs are actually being used.

To show this in action, I created a new subsystem that depends on the "org.jboss.narayana.arjunacore" that: starts the recovery manager, begins a BasicAction, adds a BasicRecord to the transaction and then commits the BasicAction. I used a subsytem to demonstrate this as I wanted something low level that would match closely enough, how Transactional MSC would use transactions.

This is just a quick prototype, I've not considered what further configuration of ArjunaCore should be done. Neither have I ensured other services, such as the transaction reaper, are running. I can work on this, if we decide this is the right approach to take (initially).

[1] https://github.com/paulrobinson/jboss-as/tree/TxMSC_arjuna_module

[dmlloyd]

Tom Jenkinson wrote:

Hi David,

Sorry, I think I may have used the wrong terminology here, what it turns out I mean is I wonder if we can prototype (achieve?) the classloader separation using two separate "JBoss Modules" in WildFly. As you have identified I would expect the same jar to be present in the two separate "modules" where the "msc" depends on an "arjunacore:main" module and "transactions subsystem" depends on the existing "jts:main" module.

Is "JBoss Modules" available at the stage that we would need it, I can appreciate that subsystems are not.

Yes this is basically in line with my thinking.  Additionally it would be ideal if the arjunacore module were as minimized as possible (not essential though).

[dmlloyd]

Paul Robinson wrote:

David Lloyd wrote:

Paul Robinson wrote:

David,

My current thoughts are that we can get classloading to do most of the heavy-lifting here and prevent too-much disruption of the Narayana code-base. Are you able to take a look at the "Overview of the solution" section and let me know if you think I am heading in the right direction? I suspect the main problem would be that we are simplifying the development work at the cost of performance and memory footprint due to there being two TMs running.

Thanks,

Paul.

One thing I want to point out here is that a key use case of the new streamlined management system is making embedded use of the server infrastructure easier (and by easier I mean "humanly possible").  So assuming we prove out this idea, the class loading solution can never be more than temporary (else I'll have to hack up some kind of infrastructure to replicate modules, which could be quite ugly - and if effort is spent I'd rather it be done to improve a thing rather than add odd one-off hacks).

I probably don't understand enough about how the "embeded use of the server infrastructure" would work, but here's my understanding:

There will be a separate project (to WildFly) containing the kernel of the AS. This will contain just enough to manage the lifecycle of installed Subsystems, provide management functionality and some other low-level things. There will then be other projects that build on this infrastructure; one such project being WildFly. In the WildFly case, a number of subsystems would be provided that when installed in the "kernel" would provide the Application Server as we currently know it. Any java libraries required by this kernel would be used just by it and not be visible in the class loaders used by the installed Subsystems.

So, assuming what I say above is mostly correct, I would see MSC in the "kernal" having a dependency on ArjunaCore with a TransactionManager and Recovery Manager initialized and running in the kernel's class loader. I would then imagine that subsystems installed in the kernel, would use JBoss Modules to manage their dependencies. In which case, any subsystem (or application) could depend on the current org.jboss.jts module if transactions which would run in a different class loader and thus be isolated from the kernel's instance of ArjunaCore.

That's reasonably accurate.  There's actually one class loader per module, but the gist of it is that yes the two instances would be fully isolated from each other.

[dmlloyd]

Paul Robinson wrote:

All,

I've knocked up a quick prototype [1] that uses JBoss Modules to provide classloader isolation. I'm sure we all (especially David) already know what this would look like, but I thought it would be a good idea to create something so that I can make sure I understand it properly and so that we have something tangible to discuss.

In the prototype, I've created a new JBossModule called "org.jboss.narayana.arjunacore". This module has its own jbossts-properties.xml (located at /modules/system/layers/base/org/jboss/narayana/arjunacore/main/config/jbossts-properties.xml) for configuring this instance of ArjunaCore. In the property file I have changed the object store location to be in "./dummymsc-object-store"; this allows us to see that two separate configs are actually being used.

To show this in action, I created a new subsystem that depends on the "org.jboss.narayana.arjunacore" that: starts the recovery manager, begins a BasicAction, adds a BasicRecord to the transaction and then commits the BasicAction. I used a subsytem to demonstrate this as I wanted something low level that would match closely enough, how Transactional MSC would use transactions.

This is just a quick prototype, I've not considered what further configuration of ArjunaCore should be done. Neither have I ensured other services, such as the transaction reaper, are running. I can work on this, if we decide this is the right approach to take (initially).

[1] https://github.com/paulrobinson/jboss-as/tree/TxMSC_arjuna_module

That's more or less what I had in mind.  Though it's likely only MSC itself which will have this dependency - and maybe the new wildfly core management system - which both lie underneath extensions, rather than being extensions themselves.  Those may well be the only things in the system that "see" arjunacore directly in fact, unless we directly expose the arjunacore APIs for coordination purposes.

[paul.robinson]

David,

David Lloyd wrote:

Paul Robinson wrote:

All,

I've knocked up a quick prototype [1] that uses JBoss Modules to provide classloader isolation. I'm sure we all (especially David) already know what this would look like, but I thought it would be a good idea to create something so that I can make sure I understand it properly and so that we have something tangible to discuss.

In the prototype, I've created a new JBossModule called "org.jboss.narayana.arjunacore". This module has its own jbossts-properties.xml (located at /modules/system/layers/base/org/jboss/narayana/arjunacore/main/config/jbossts-properties.xml) for configuring this instance of ArjunaCore. In the property file I have changed the object store location to be in "./dummymsc-object-store"; this allows us to see that two separate configs are actually being used.

To show this in action, I created a new subsystem that depends on the "org.jboss.narayana.arjunacore" that: starts the recovery manager, begins a BasicAction, adds a BasicRecord to the transaction and then commits the BasicAction. I used a subsytem to demonstrate this as I wanted something low level that would match closely enough, how Transactional MSC would use transactions.

This is just a quick prototype, I've not considered what further configuration of ArjunaCore should be done. Neither have I ensured other services, such as the transaction reaper, are running. I can work on this, if we decide this is the right approach to take (initially).

[1] https://github.com/paulrobinson/jboss-as/tree/TxMSC_arjuna_module

That's more or less what I had in mind.  Though it's likely only MSC itself which will have this dependency - and maybe the new wildfly core management system - which both lie underneath extensions, rather than being extensions themselves.  Those may well be the only things in the system that "see" arjunacore directly in fact, unless we directly expose the arjunacore APIs for coordination purposes.

Yes, I was assuming that this would lie underneath extensions. I just needed some place to do the prototype and I felt that deploying an application would be too high up the stack.

Do you agree that this approach is adequate for you to develop a solution for Transactional MSC? Also, does it look like it would be adequate for how you are planning on implementing the "embeded use of the server infrastructure" feature? If so, I think we can defer the discussion around removing global state, until we are further down the implementation path of Transactional MSC.

If so, I think the next steps are for me to update my prototype to create a minimal configuration of ArjunaCore that meets your requirements, as documented on this page so far. For example, disabling stats, disabling the recovery manager's network port, etc.

Should we also document, in more detail, your plan on how you intend to use ArjunaCore? This would probably make it easier for us to provide feedback and to also make sure Narayana is providing everything you need.