limiting resouces usage in FUSE MB 5.0 (ActiveMQ 5.0)
tamirw Feb 7, 2008 9:20 AMHi,
We're evaluating Fuse MB 5.0 as our messaging framework.
In the process of evaluation, a couple of question arose which I haven't been able to answer by myself.
Our environment consist of a lot of remote producers and one main consumer.
We have strict requirements on resources usage on the producer machines and I haven't found the way to enforce those using MB. The requirements are :
a) The memory usage of the broker cannot exceed a pre-defined limit - since the messages should be persisted anyway, and they would never be consumed from that broker, there is no reason to keep any messages in RAM. All should be persisted as soon as possible.
This is not so much of a problem cause our testing show that the memory usage is controlled somehow (e.g it doesn't explode even after producing a lot of data). However, I didn't find the place to configure that limit. (none of the configuration - both java and xml) had any effect on the memory usage.
b) The disk space taken by the AMQStore (journal and data files) should also be limited. When the threshold is reached, old messages should be discarded from the store to make place to newer ones. Again, none of my configuration efforts worked, and the I ended up with huge files on the file system.
I'm also attaching the .xml configuration file I used for my testing. Some of it is remarked (as a result of many trials) but I'm sure you'll understand what I was trying to do.
If I'm misusing it or misconfiguring it - please let me know.
Any ideas would be greatly appreciated.
Tamir
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.org/config/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://activemq.org/config/1.0 http://activemq.apache.org/schema/activemq-core.xsd http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring.xsd"> <!-- Allows us to use system properties as variables in this configuration file --> <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"></bean> <broker xmlns="http://activemq.org/config/1.0" brokerName="localhost" deleteAllMessagesOnStartup="true"> <!-- The transport connectors ActiveMQ will listen to --> <transportConnectors> <transportConnector name="openwire" uri="tcp://localhost:61616" discoveryUri="multicast://default"></transportConnector> <!-- <transportConnector name="ssl" uri="ssl://localhost:61617"/> <transportConnector name="stomp" uri="stomp://localhost:61613"></transportConnector> <transportConnector name="xmpp" uri="xmpp://localhost:61222"></transportConnector> --> </transportConnectors> <!-- The store and forward broker networks ActiveMQ will listen to --> <networkConnectors> <!-- by default just auto discover the other brokers --> <!-- <networkConnector name="host1 and host2" uri="static://(tcp://host1:61616,tcp://host2:61616)"/> <networkConnector name="default-nc" uri="multicast://default"></networkConnector> --> </networkConnectors> <!-- Use the following if you wish to configure the journal with JDBC --> <!-- <persistenceAdapter> <journaledJDBC journalLogFiles="5" dataDirectory="${activemq.base}/activemq-data" dataSource="#postgres-ds"></journaledJDBC> </persistenceAdapter> --> <!-- Or if you want to use pure JDBC without a journal --> <!-- <persistenceAdapter> <jdbcPersistenceAdapter dataSource="#postgres-ds"></jdbcPersistenceAdapter> </persistenceAdapter> --> <!-- Use the following to set the broker memory limit --> <persistenceAdapter> <amqPersistenceAdapter maxFileLength="7mb"> <usageManager> <systemUsage> <memoryUsage> <memoryUsage limit="10 mb" percentUsageMinDelta="20"></memoryUsage> </memoryUsage> <tempUsage> <tempUsage limit="100 mb"></tempUsage> </tempUsage> <storeUsage> <storeUsage limit="15 mb" name="foo2"></storeUsage> </storeUsage> </systemUsage> </usageManager> </amqPersistenceAdapter> </persistenceAdapter> <producerSystemUsage> <systemUsage> <memoryUsage> <memoryUsage limit="10 mb" percentUsageMinDelta="20"></memoryUsage> </memoryUsage> <tempUsage> <tempUsage limit="100 mb"></tempUsage> </tempUsage> <storeUsage> <storeUsage limit="15 mb" name="foo1"></storeUsage> </storeUsage> </systemUsage> </producerSystemUsage> <!-- Use the following to configure how ActiveMQ is exposed in JMX <managementContext> <managementContext connectorPort="1099" jmxDomainName="org.apache.activemq"></managementContext> </managementContext> --> </broker> <camelContext id="camel" xmlns="http://activemq.apache.org/camel/schema/spring"> <!-- You can use a <package> element for each root package to search for Java routes --> <package>org.foo.bar</package> <!-- You can use Spring XML syntax to define the routes here using the <route> element --> <route> <from uri="activemq:example.A"></from> <to uri="activemq:example.B"></to> </route> </camelContext> <!-- lets create a command agent to respond to message based admin commands on the ActiveMQ.Agent topic --> <commandAgent xmlns="http://activemq.org/config/1.0"></commandAgent> <!-- An embedded servlet engine for serving up the Admin console --> <jetty xmlns="http://mortbay.com/schemas/jetty/1.0"> <connectors> <nioConnector port="8161"></nioConnector> </connectors> <handlers> <webAppContext contextPath="/admin" resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"></webAppContext> <webAppContext contextPath="/demo" resourceBase="${activemq.base}/webapps/demo" logUrlOnStart="true"></webAppContext> </handlers> </jetty> <!-- This xbean configuration file supports all the standard spring xml configuration options --> <!-- Postgres DataSource Sample Setup --> <!-- <bean id="postgres-ds" class="org.postgresql.ds.PGPoolingDataSource"> <property name="serverName" value="localhost"></property> <property name="databaseName" value="activemq"></property> <property name="portNumber" value="0"></property> <property name="user" value="activemq"></property> <property name="password" value="activemq"></property> <property name="dataSourceName" value="postgres"></property> <property name="initialConnections" value="1"></property> <property name="maxConnections" value="10"></property> </bean> --> <!-- MySql DataSource Sample Setup --> <!-- <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"> <property name="driverClassName" value="com.mysql.jdbc.Driver"></property> <property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"></property> <property name="username" value="activemq"></property> <property name="password" value="activemq"></property> <property name="poolPreparedStatements" value="true"></property> </bean> --> <!-- Oracle DataSource Sample Setup --> <!-- <bean id="oracle-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"> <property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"></property> <property name="url" value="jdbc:oracle:thin:@localhost:1521:AMQDB"></property> <property name="username" value="scott"></property> <property name="password" value="tiger"></property> <property name="poolPreparedStatements" value="true"></property> </bean> --> <!-- Embedded Derby DataSource Sample Setup --> <!-- <bean id="derby-ds" class="org.apache.derby.jdbc.EmbeddedDataSource"> <property name="databaseName" value="derbydb"></property> <property name="createDatabase" value="create"></property> </bean> --> </beans>
Edited by: tamirw on Feb 7, 2008 9:19 AM