Programmatic configuration of jvmRoute

[brian.stansberry]

JBoss Messaging has introduced a requirement that all nodes in a cluster be given a unique id (see http://jira.jboss.com/jira/browse/JBMESSAGING-1121). Since we now have the requirement I'd like to brainstorm a bit about how that id could be used across all cases in the AS where we want to uniquely identify nodes.

The obvious one that comes to mind is the Tomcat jvmRoute.

For now, let's assume that the JBM unique id is represented by system property "server.peer.id". (Also assume we'll come up a more generic name for the property.)

A simple thing to do would be to hard code the following into the std server.xml:

<Engine name="jboss.web" defaultHost="localhost" jvmRoute="${server.peer.id:0}">

Downside to that is we now dot append the id (or default 0) to every session id, even if we aren't using JK. That's probably no big deal; anyone see a reason why it would be?

If it is a problem, perhaps we could leave the jvmRoute attribute out of server.xml and programatically call Engine.setJvmRoute() if the TomcatDeployer's useJK property is set to true. That seems pretty clunky though.

Note that JBM requires that the id be a non-negative integer.

[starksm64]

It needs to be part of the container metadata as in general this should be coming from some server/clustered configuration potentially set by the admin tools. A tomcat deployer specific setting is just an override to a higher level server setting.

In general we need to forget that xml exists, and define a metadata model that encapsulates all externalized behavior, and then map the sources of that information onto the metadata model. The tomcat deployer is a long way from correctly using such a model due to its internal parsing of its own descriptors like server.xml, conf/web.xml, etc. Its a todo to have jbossweb completely driven by the web container metadata model.

[brian.stansberry]

OK, with that approach presumably you could have some logic somewhere that says "if my metadata says I'm using JK, then check for jvmRoute metadata" which defaults to the overall server id.

Is having jbossweb completely driven by web container metadata something you're thinking about for AS 5.0.0? If not, then my original question about whether it's OK to add a std jvmRoute attribute to server.xml still stands.

[brian.stansberry]

This leads into the general topic of what the use cases are for identifying a server and what type of metadata could satisfy each use case; goal being to see if we can have a single value per node, or no value at all. I opened a thread on this a while ago but got no response. Scott/Tim, etc., any inputs you have on http://www.jboss.com/index.html?module=bb&op=viewtopic&t=108609 would be appreciated. Tim Fox is thinking about approaches that would avoid having an externalized node id, but if we need to have one anyway for other use cases, maybe he doesn't need to bother.

[starksm64]

OK, with that approach presumably you could have some logic somewhere that says "if my metadata says I'm using JK, then check for jvmRoute metadata" which defaults to the overall server id.

Is having jbossweb completely driven by web container metadata something you're thinking about for AS 5.0.0? If not, then my original question about whether it's OK to add a std jvmRoute attribute to server.xml still stands.

No, it won't be completely rewritten by that timeframe. Its ok to refer to a system property for now. In the future there should be a jvmRoute setting that could default to the system property if not set. This ultimately would be a lookup against the metadata repository to see if had been mapped onto another value, coming from deployment metadata, annotations, system property, server settings, etc.

[jfclere]

Don't forget that we have actually no ways to forward this information to mod_jk/mod_proxy so changing the jvmRoute programmaticaly requires complex modifications there too.

[brian.stansberry]

If it were changed programatically (which, given the earlier part of the thread, is very probably not necessary) it would just be as part of the startup sequence of JBossWeb, before the AJP connector is even started. E.g.

TomcatDeployer does what it does now which results in parsing server.xml, Engine object being created.

Added step: TomcatDeployer sees it's configured with useJK=true, sees Engine.getJvmRoute() == null, so it creates a jvmRoute value based on the global server Id and calls Engine.setJvmRoute().

Existing process continues: TomcatDeployer.startConnectors() gets called when the signal comes in that the overall AS start is done.

Previously unstated: the user would of course have to know how JBossWeb is going to convert the server Id into a jvmRoute, and use the correct worker name in workers.properties.

[rmaucher]

I see you're thinking about this from your own perspective, but based on the multitude of AJP connectors we support, it is simply not going to be possible to set the value of jvmRoute dynamically, and that's not going to change. I suggest to simply give up on the whole idea: the value of the jvmRoute attribute must remain static.

[brian.stansberry]

OK; that's why I asked. :-)

Do you see any significant issue with making something like this the default in server.xml?

<Engine name="jboss.web" defaultHost="localhost" jvmRoute="n${server.peer.id:0}">

Assumption here is that JBM's needs force us to set server.peer.id to some integer value.

Downside I see is just a minor performance hit of needlessly adding ".n0" to session ids when JK isn't used. Anything worse?