Node identification

[brian.stansberry]

Tim Fox and I were having a discussion on http://jira.jboss.com/jira/browse/JBMESSAGING-957 about how to give identity to servers in a cluster. Wanted to bring it here to get inputs from others who may face similar issues and in particular from the JON folks.

Issue is basically that there are various places where code needs to give some kind of identity to a server:

1) Tomcat engine if mod_jk is used (jvmRoute attribute).
2) JBoss Messaging if clustered.
3) HAPartition to prevent two incarnations of the same server joining the cluster.

Tim and I are discussing setting the name for a server via run.sh, perhaps using something like what's done with $JBOSS_HOST and -b in jboss_init_redhat.sh:

JBOSS_NODE_NAME=${JBOSS_NODE_ID:+"-i $JBOSS_NODE_ID"}
#define the script to use to start jboss
JBOSSSH=${JBOSSSH:-"$JBOSS_HOME/bin/run.sh -c $JBOSS_CONF $JBOSS_BIND_ADDR $JBOSS_NODE_ID"}

The above presumes the addition of a new startup switch '-i' or --id; a system property could be used as well.

Basically just wanted to get any comments any one had adding this sort of thing, e.g.

1) what are your use cases for wanting an id for a server?
2) does it need to be persistent across restarts (i.e. can't be randomly generated at runtime or something)?
3) Any constraints on the id? E.g. JBM wants an int id, which isn't really appropriate for the jvmRoute or HAPartition use cases.
4) Any thoughts on setting it based on the local env via the startup script? Would the JON folks want to do it some other way?

[brian.stansberry]

Bump. This issue has come up again. See http://jira.jboss.com/jira/browse/JBMESSAGING-1121
and http://www.jboss.com/index.html?module=bb&op=viewtopic&t=122241 .

[starksm64]

Random without some stable alias is not useful for debugging. In general we should have a set of server metadata that can for which system properties would be one source of specifying values. As the server becomes more managable, it could be admin tools that provide the values. The starting point should be the ServerConfig bean. This should be a managed object that includes settings for network interfaces, virtual hosts, cluster ids, service ports, etc. Its then an issue of wiring to the subsystems that need this information. Either they have integration code that is aware of the ServerConfig metadata, or there is integration code that delegates to the server metadata repository so that a piece of metadata like a jvm route can be mapped onto a ServerConfig setting.

package org.jboss.system.server;

public interface ServerConfig
{

 /** The default partition name */
 String DEFAULT_PARITION_NAME = "DefaultPartition";

 /** The partition name property */
 String PARTITION_NAME_PROPERTY = "jboss.partition.name";

 /** The udp address property */
 String PARTITION_UDP_PROPERTY = "jboss.partition.udpGroup";

 /** The udp port property */
 String PARTITION_UDP_PORT_PROPERTY = "jgroups.udp.mcast_port";

 /** Whether to load native libraries */
 String NATIVE_LOAD_PROPERTY = "jboss.native.load";

 /** The location of native libraries property */
 String NATIVE_DIR_PROPERTY = "jboss.native.dir";

 /////////////////////////////////////////////////////////////////////////
 // Bootstrap Specific Config //
 /////////////////////////////////////////////////////////////////////////

 /**
 * Constant that holds the name of the environment property
 * for specifying a comma seperated list of the basenames of
 * to the boot libraries required load the core system.
 *
 * <p>These libraries will be loaded from <tt>LIBRARY_URL</tt>.
 */
 String BOOT_LIBRARY_LIST = "jboss.boot.library.list";

 /**
 * Constant that holds the name of the environment property
 * for specifying the class type of the server to create.
 */
 String SERVER_TYPE = "jboss.server.type";

 /**
 * The bootstrap url
 *
 * <p>If not set then the server will default to {@link #DEFAULT_ROOT_DEPLOYMENT_FILENAME}.
 */
 String BOOTSTRAP_URL = "jboss.bootstrap.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the root deployment filename (relative to the server
 * config URL that will be deployed to finalize the boot strap process.
 *
 * <p>If not set then the server will default to {@link #DEFAULT_ROOT_DEPLOYMENT_FILENAME}.
 */
 String ROOT_DEPLOYMENT_FILENAME = "jboss.server.root.deployment.filename";


 /////////////////////////////////////////////////////////////////////////
 // Configuration Value Identifiers //
 /////////////////////////////////////////////////////////////////////////

 /**
 * Constant that holds the name of the environment property
 * for specifying the home directory for JBoss.
 */
 String HOME_DIR = "jboss.home.dir";

 /**
 * Constant that holds the name of the environment property
 * for specifying the home URL for JBoss.
 *
 * <p>If not set then the value of HOME_DIR will converted into a URL.
 */
 String HOME_URL = "jboss.home.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the URL where JBoss will read library files
 * from.
 *
 * <p>Defaults to <tt><em>HOME_URL</em>/lib</tt>/
 */
 String LIBRARY_URL = "jboss.lib.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the URL where JBoss will read patch library files
 * from.
 *
 * <p>If this value is a <tt>file</tt> URL, then all .zip and .jar files
 * inside will be prepended to the classpath. Otherwise the URL will be
 * added to the classpath. If not set then the no patch files will be
 * loaded.
 */
 String PATCH_URL = "jboss.patch.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the name of the server which will be used to
 * calculate the servers home directory and url.
 *
 * <p>Defaults to <tt>default</tt>.
 */
 String SERVER_NAME = "jboss.server.name";

 /**
 * Constant that holds the name of the environment property
 * for specifying the base directory for calculating server
 * home directories.
 *
 * <p>Defaults to <tt><em>HOME_DIR</em>/server</tt>.
 */
 String SERVER_BASE_DIR = "jboss.server.base.dir";

 /**
 * Constant that holds the name of the environment property
 * for specifying the server home directory for JBoss.
 *
 * <p>Defaults to <tt><em>SERVER_BASE_DIR</em>/<em>SERVER_NAME</em></tt>.
 */
 String SERVER_HOME_DIR = "jboss.server.home.dir";

 /**
 * Constant that holds the name of the environment property
 * for specifying the server log directory for JBoss.
 *
 * <p>Defaults to <tt><em>SERVER_HOME_DIR</em>/<em>log</em></tt>.
 */
 String SERVER_LOG_DIR = "jboss.server.log.dir";

 /**
 * Constant that holds the name of the environment property
 * for specifying the directory which JBoss will use for
 * temporary file storage.
 *
 * <p>Defaults to <tt><em>SERVER_HOME_DIR</em>/tmp</tt> .
 */
 String SERVER_TEMP_DIR = "jboss.server.temp.dir";

 /**
 * Constant that holds the name of the environment property
 * for specifying the directory which JBoss will use for
 * persistent data file storage.
 *
 * <p>Defaults to <tt><em>SERVER_HOME_DIR</em>/data</tt>.
 */
 String SERVER_DATA_DIR = "jboss.server.data.dir";

 /**
 * Constant that holds the name of the environment property
 * for specifying the base URL for calculating server
 * home URLs.
 *
 * <p>Defaults to <tt><em>HOME_URL</em>/server</tt>.
 */
 String SERVER_BASE_URL = "jboss.server.base.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the server home URL for JBoss.
 *
 * <p>Defaults to <tt><em>SERVER_BASE_URL</em>/<em>SERVER_NAME</em></tt>.
 */
 String SERVER_HOME_URL = "jboss.server.home.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the server configuration URL.
 *
 * <p>Defaults to <tt><em>SERVER_HOME_UTL</em>/conf</tt> .
 */
 String SERVER_CONFIG_URL = "jboss.server.config.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the URL where JBoss will read server specific
 * library files from.
 *
 * <p>Defaults to <tt><em>SERVER_HOME_URL</em>/lib</tt>/
 */
 String SERVER_LIBRARY_URL = "jboss.server.lib.url";

 /**
 * Constant that holds the name of the environment property
 * for specifying the bind address for all jboss services
 *
 */
 String SERVER_BIND_ADDRESS = "jboss.bind.address";

 /**
 * Constant that holds the name of the environment property
 * for specifying whether or not the server should exit the
 * JVM on shutdown.
 *
 * <p>If not set then the server will default to exiting on shutdown.
 */
 String EXIT_ON_SHUTDOWN = "jboss.server.exitonshutdown";

 /**
 * Constant that holds the name of the environment property for
 * specifying whether or not the server should shutdown
 * synchronously (true) or asynchronously (false).
 *
 * <p>If not set then the server will default to asynchronous shutdown.
 */
 String BLOCKING_SHUTDOWN = "jboss.server.blockingshutdown";

 /**
 * Constant that holds the name of the environment property
 * for specifying whether or not to install the lifethread
 *
 * <p>If not set then the server will default to installing the lifethread.
 */
 String INSTALL_LIFE_THREAD = "jboss.server.lifethread";

 /**
 * Constant that holds the name of the environment property for
 * specifying whether or not the server should log and ignore
 * exceptions when setting the URLStreamHandlerFactory.
 *
 * <p>If not set then the server will default to asynchronous shutdown.
 */
 String REQUIRE_JBOSS_URL_STREAM_HANDLER_FACTORY = "jboss.server.requirejbossurlstreamhandlerfactory";

 /**
 * Constant that holds the name of the environment property for specifying
 * whether or not to use as the main jboss server the MBeanServer returned
 * from ManagementFactory.getPlatformMBeanServer(), when running under jdk1.5+
 *
 * <p>If not set then jboss will instantiate its own MBeanServer
 */
 String PLATFORM_MBEANSERVER = "jboss.platform.mbeanserver";

 /////////////////////////////////////////////////////////////////////////
 // Path Suffixes //
 /////////////////////////////////////////////////////////////////////////

 /**
 * The suffix used when generating the default value for {@link #LIBRARY_URL}
 * and {@link #SERVER_LIBRARY_URL}.
 */
 String LIBRARY_URL_SUFFIX = "lib/";

 /**
 * The suffix used when generating the default value for {@link #SERVER_CONFIG_URL}.
 */
 String SERVER_CONFIG_URL_SUFFIX = "conf/";

 /**
 * The suffix used when generating the default value for {@link #SERVER_BASE_DIR}.
 */
 String SERVER_BASE_DIR_SUFFIX = "server";

 /**
 * The suffix used when generating the default value for {@link #SERVER_BASE_URL}.
 */
 String SERVER_BASE_URL_SUFFIX = "server/";

 /**
 * The suffix used when generating the default value for {@link #SERVER_DATA_DIR}.
 */
 String SERVER_DATA_DIR_SUFFIX = "data";

 /**
 * The suffix used when generating the default value for {@link #SERVER_LOG_DIR}.
 */
 String SERVER_LOG_DIR_SUFFIX = "log";

 /**
 * The suffix used when generating the default value for {@link #SERVER_TEMP_DIR}.
 */
 String SERVER_TEMP_DIR_SUFFIX = "tmp";


 /////////////////////////////////////////////////////////////////////////
 // Defaults //
 /////////////////////////////////////////////////////////////////////////

 /** The default value for {@link #SERVER_NAME}. */
 String DEFAULT_SERVER_NAME = "default";

 /** The default value for {@link #EXIT_ON_SHUTDOWN}. */
 boolean DEFAULT_EXIT_ON_SHUTDOWN = true;

 /** The default value for {@link #BLOCKING_SHUTDOWN}. */
 boolean DEFAULT_BLOCKING_SHUTDOWN = false;

 /** The default value for {@link #INSTALL_LIFE_THREAD}. */
 boolean DEFAULT_INSTALL_LIFE_THREAD = true;

 /** The default value for {@link #REQUIRE_JBOSS_URL_STREAM_HANDLER_FACTORY}. */
 boolean DEFAULT_REQUIRE_JBOSS_URL_STREAM_HANDLER_FACTORY = true;

 /** The default value for {@link #PLATFORM_MBEANSERVER}. */
 boolean DEFAULT_PLATFORM_MBEANSERVER = false;

 /** The default value for {@link #ROOT_DEPLOYMENT_FILENAME}. */
 String DEFAULT_ROOT_DEPLOYMENT_FILENAME = "jboss-service.xml";


 /////////////////////////////////////////////////////////////////////////
 // Typed Access Methods //
 /////////////////////////////////////////////////////////////////////////

 /**
 * Get the bootstrap url
 *
 * @return the bootstrap url
 */
 URL getBootstrapURL();

 /**
 * Get the local home directory which the server is running from.
 *
 * @return The local server home directory.
 */
 File getHomeDir();

 /**
 * Get the home URL which the server is running from.
 *
 * @return The home URL which the server is running from.
 */
 URL getHomeURL();

 /**
 * Get the library URL for the server.
 *
 * @return The library URL for the server.
 */
 URL getLibraryURL();

 /**
 * Get the patch URL for the server.
 *
 * @return The patch URL for the server.
 */
 URL getPatchURL();

 /**
 * Get the name of the server.
 *
 * @return The name of the server.
 */
 String getServerName();

 /**
 * Get the base directory for calculating server home directories.
 *
 * @return Base server home directory.
 */
 File getServerBaseDir();

 /**
 * Get the server home directory.
 *
 * @return Server home directory.
 */
 File getServerHomeDir();

 /**
 * Get the directory where log files will be stored.
 *
 * @return The directory where the server writes log files.
 */
 File getServerLogDir();

 /**
 * Get the directory where temporary files will be stored.
 *
 * @return The directory where the server stores temporary files.
 */
 File getServerTempDir();

 /**
 * Get the directory where local data will be stored.
 *
 * @return The directory where the server stores local data.
 */
 File getServerDataDir();

 /**
 * Get the base directory for calculating server home URLs.
 *
 * @return Base server home URL.
 */
 URL getServerBaseURL();

 /**
 * Get the server home URL.
 *
 * @return Server home URL.
 */
 URL getServerHomeURL();

 /**
 * Get the server library URL.
 *
 * @return Server library URL.
 */
 URL getServerLibraryURL();

 /**
 * Get the server configuration URL.
 *
 * @return Server configuration URL.
 */
 URL getServerConfigURL();

 /**
 * Get the current value of the flag that indicates if we are
 * using the platform MBeanServer as the main jboss server.
 * Both the {@link ServerConfig#PLATFORM_MBEANSERVER}
 * property must be set, and the jvm must be jdk1.5+
 *
 * @return true if jboss runs on the jvm platfrom MBeanServer
 */
 boolean getPlatformMBeanServer();

 /**
 * Enable or disable exiting the JVM when {@link Server#shutdown()} is called.
 * If enabled, then shutdown calls {@link Server#exit()}. If disabled, then
 * only the shutdown hook will be run.
 *
 * @param flag True to enable calling exit on shutdown.
 */
 void setExitOnShutdown(boolean flag);

 /**
 * Get the current value of the exit on shutdown flag.
 *
 * @return The current value of the exit on shutdown flag.
 */
 boolean getExitOnShutdown();


 /**
 * Get the BlockingShutdown value.
 * @return the BlockingShutdown value.
 */
 boolean getBlockingShutdown();

 /**
 * Set the BlockingShutdown value.
 * @param blockingShutdown The new BlockingShutdown value.
 */
 void setBlockingShutdown(boolean blockingShutdown);

 /**
 * Whether to install the lifethread
 *
 * @return true to install the life thread
 */
 boolean isInstallLifeThread();

 /**
 * Get the RequireJBossURLStreamHandlerFactory value.
 * @return the RequireJBossURLStreamHandlerFactory value.
 */
 boolean getRequireJBossURLStreamHandlerFactory();

 /**
 * Set the RequireJBossURLStreamHandlerFactory value.
 * @param requireJBossURLStreamHandlerFactory The new RequireJBossURLStreamHandlerFactory value.
 */
 void setRequireJBossURLStreamHandlerFactory(boolean requireJBossURLStreamHandlerFactory);

 /**
 * Set the filename of the root deployable that will be used to finalize
 * the bootstrap process.
 *
 * @param filename The filename of the root deployable.
 */
 void setRootDeploymentFilename(String filename);

 /**
 * Get the filename of the root deployable that will be used to finalize
 * the bootstrap process.
 *
 * @return The filename of the root deployable.
 */
 String getRootDeploymentFilename();

 /**
 * Get the native dir for unpacking
 *
 * @return the directory
 */
 File getServerNativeDir();

 /**
 * Get the temporary deployment dir for unpacking
 *
 * @return the directory
 */
 File getServerTempDeployDir();

 /**
 * Get the server Specification-Version
 *
 * @return the spec version
 */
 String getSpecificationVersion();
}

[brian.stansberry]

Random without some stable alias is not useful for debugging.

Is this in reference to the concept discussed on JBMESSAGING-1121 of using a GUID?


Re: getting the data from ServerConfig, etc., sounds good to me. That gets into where the metadata comes from / how services consume it. My immediate concern is what the metadata *is* -- who needs what, and in what form. Assumption being that if end users are somehow or other going to have to configure this, we should make their lives as easy as possible by trying to settle on a single base piece of information. Maybe services massage that base information or allow per-service overrides, but there is just one base value.

My original post on this thread stated that the integer id JBM was using "isn't really appropriate for the jvmRoute or HAPartition use cases." I don't know why I wrote that; can't think of any reason why an integer is inappropriate. mod_jk may force us to prefix the id with some string to make a jvmRoute, but that's no big deal to work around (e.g. in a workers.properties file 'worker.0.port=8009' *might* be an issue but 'worker.node0.port=8009' is fine.)

At this point the known use cases are:

1) JBM -- an id is required and must be an integer (unless Tim decides otherwise).

2) JBossWeb -- id only required when JK is used. Integer is OK, minor preference that it's not a really long one.

3) HAPartition. Required for a marginal use case of preventing a node joining the cluster twice. Integer is OK.

Anything else? What about JON?

[starksm64]

Is this in reference to the concept discussed on JBMESSAGING-1121 of using a GUID?

Having a jboss.server.guid that maps to a ServerConfig.guid is fine, it just should not be the root your talking about.

Having anything but a string be the base doesn't lend itself well to reuse. The JON team needs to define what the aggregate management views are, and what needs to be in the logical ServerConfig view to support this. Having the following seems like a no-brainer:

String serverId; - unique within management domain, defaults to hostname
String[] serverAliases; - non-unique aliases
GUID serverGuid; - supposedly universally unique. The only question is permanence of this value. Could certainly be generated if not found and reused if found. With jdk6 you can get the hardware mac address as input for this.

[brian.stansberry]

A string serverId could work fine for jvmRoute and HAPartition. The default hostname value would be no good if you had a vertical cluster or were concerned about leaking your hostnames as part of a web session id. But then you just have to set it to something else.

A GUID might work for JBM; Tim is looking into it. AIUI, basic issue is the type of the value needs to be performant in database queries.

[brian.stansberry]

David Lloyd is looking to actually get something done on this general issue, so discussion has continued at http://community.jboss.org/thread/148809?tstart=0