Differences Between the Express and Flex JBossAS7 Configurations
Posted by starksm64 in Scott Stark's Blog on Aug 10, 2011 12:20:36 PMBoth the Express and Flex cartridges are based on the JBossAS7 web profile. Express reduces some of the services enabled by default in the web profile as I discussed in this post, while Flex actually adds addition services for clustering of web sessions using Infinispan and JGroups.
Web Session Clustering
The configuration of the web session cache replication employs a configuration that utilizes a TCP based JGroups configuration, and a FILE_PING discovery protocol that utilizes the Flex underlying clustered file system based on the glusterfs. The reason for use of TCP is that multicast is not usable in the Amazon cloud environment, and the reason the FILE_PING discovery protocol is used is that the Flex layer is in control of the cluster membership. By writing the cluster membership to the application shared directory, Flex is in control of what nodes an application sees. The Infinispan and JGroups subsystem configuration fragments are listed here:
<subsystem xmlns="urn:jboss:domain:infinispan:1.0" default-cache-container="hibernate">
<cache-container name="hibernate" default-cache="local-query">
<local-cache name="entity">
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</local-cache>
<local-cache name="local-query">
<eviction strategy="LRU" max-entries="10000"/>
<expiration max-idle="100000"/>
</local-cache>
<local-cache name="timestamps">
<eviction strategy="NONE"/>
</local-cache>
</cache-container>
<!-- web session replication cache definitions -->
<cache-container name="web" default-cache="repl">
<alias>standard-session-cache</alias>
<transport stack="tcp" />
<replicated-cache name="repl" mode="ASYNC" batching="true">
<locking isolation="REPEATABLE_READ"/>
<file-store/>
</replicated-cache>
<distributed-cache name="dist" mode="ASYNC" batching="true">
<locking isolation="REPEATABLE_READ"/>
<file-store/>
</distributed-cache>
</cache-container>
</subsystem>
<subsystem xmlns="urn:jboss:domain:jgroups:1.0" default-stack="tcp">
<stack name="tcp">
<transport type="TCP" socket-binding="jgroups-tcp" diagnostics-socket-binding="jgroups-diagnostics"/>
<protocol type="FILE_PING">
<property name="location">${vostok.app.shared}</property>
<property name="timeout">5000</property>
<property name="num_initial_members">1</property>
</protocol>
<protocol type="MERGE2"/>
<protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd"/>
<protocol type="FD"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="BARRIER"/>
<protocol type="pbcast.NAKACK"/>
<protocol type="UNICAST"/>
<protocol type="pbcast.STABLE"/>
<protocol type="VIEW_SYNC"/>
<protocol type="pbcast.GMS"/>
<protocol type="UFC"/>
<protocol type="MFC"/>
<protocol type="FRAG2"/>
<protocol type="pbcast.STREAMING_STATE_TRANSFER"/>
<protocol type="pbcast.FLUSH"/>
</stack>
</subsystem>
The ${vostok.app.shared} value is an application specific directory that is shared across the cluster members assocaited with the application. The Flex environment maintains the application distributed directory for each application.
Server Isolation
Both the Flex and Express environments launch a JBossAS7 instance for the logical PAAS application. In the current implementations, the mechanism by which the JBossAS7 server instances are isolated on a given PAAS envionrment EC2 server instance differ due to differences in the current Flex and Express implementations.
Isolation by Interface
The Express environment assigns each application a unique IP address on the loopback interface. It is a property of Fedora/RHEL that one can bind unique loopback addresses without having to configure a specific alias. When the AS7 cartridge is called to configure an application, it locates the next unique loopback interface IP address, and set the {ip} placeholder in the following standalone.xml fragement with the IP address:
<server xmlns="urn:jboss:domain:1.0">
<management>
...
<interfaces>
<interface name="management">
<loopback-address value="{ip}"/>
</interface>
<interface name="public">
<loopback-address value="{ip}"/>
</interface>
</interfaces>
</server>
Isolation by Ports
In the Flex environment, there is no support currently available for assigning the cartridge a unique loopback address, so what the Flex cartridge does is to determine a unique port offset based on the available of a free http port. It starts at the default value of 8080, and advances by 100 until a free port is found. The offset is then used as the port-offset value socket-binding-group configuration. The offset also has to be applied separately to each of the managment-interfaces ports due to issue: (https://issues.jboss.org/browse/AS7-1476). Note that this configuration fragment also show the use of the eth0 interface by the JGroups port references shown in the above JGroups fragment. The eth0 interface is usable interface between the instances in the Flex cluster.
<server xmlns="urn:jboss:domain:1.0">
<management>
...
<management-interfaces>
<native-interface interface="management" port="9999" />
<http-interface interface="management" port="9990"/>
</management-interfaces>
</management>
<interfaces>
<interface name="management">
<inet-address value="127.0.0.1"/>
</interface>
<interface name="public">
<inet-address value="127.0.0.1"/>
</interface>
<interface name="eth0">
<nic name="eth0"/>
</interface>
</interfaces>
<socket-binding-group name="standard-sockets" default-interface="public" port-offset='0'>
...
<socket-binding name="jgroups-tcp" port="7600" interface="eth0" />
<socket-binding name="jgroups-tcp-fd" port="57600" interface="eth0" />
...
</socket-binding-group>
</server>
Comments