5 Replies Latest reply on Feb 12, 2008 8:27 AM by tamirw

    limiting resouces usage in FUSE MB 5.0 (ActiveMQ 5.0)

    tamirw

      Hi,

       

      We're evaluating Fuse MB 5.0 as our messaging framework.

      In the process of evaluation, a couple of question arose which I haven't been able to answer by myself.

       

      Our environment consist of a lot of remote producers and one main consumer.

      We have strict requirements on resources usage on the producer machines and I haven't found the way to enforce those using MB. The requirements are :

      a) The memory usage of the broker cannot exceed a pre-defined limit - since the messages should be persisted anyway, and they would never be consumed from that broker, there is no reason to keep any messages in RAM. All should be persisted as soon as possible.

      This is not so much of a problem cause our testing show that the memory usage is controlled somehow (e.g it doesn't explode even after producing a lot of data). However, I didn't find the place to configure that limit. (none of the configuration - both java and xml) had any effect on the memory usage.

      b) The disk space taken by the AMQStore (journal and data files) should also be limited. When the threshold is reached, old messages should be discarded from the store to make place to newer ones. Again, none of my configuration efforts worked, and the I ended up with huge files on the file system.

       

      I'm also attaching the .xml configuration file I used for my testing. Some of it is remarked (as a result of many trials) but I'm sure you'll understand what I was trying to do.

      If I'm misusing it or misconfiguring it - please let me know.

       

      Any ideas would be greatly appreciated.

       

      Tamir

       

      <beans xmlns="http://www.springframework.org/schema/beans"
           xmlns:amq="http://activemq.org/config/1.0"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="http://www.springframework.org/schema/beans     http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
        http://activemq.org/config/1.0 http://activemq.apache.org/schema/activemq-core.xsd
        http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">
      
           <!-- Allows us to use system properties     as variables in     this configuration file     -->
           <bean
                class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"></bean>
      
           <broker     xmlns="http://activemq.org/config/1.0"
                brokerName="localhost" deleteAllMessagesOnStartup="true">
      
      
                <!-- The transport connectors ActiveMQ will listen to -->
                <transportConnectors>
                     <transportConnector name="openwire"
                          uri="tcp://localhost:61616" discoveryUri="multicast://default"></transportConnector>
                     <!--
                          <transportConnector name="ssl"        uri="ssl://localhost:61617"/>
                          <transportConnector name="stomp"   uri="stomp://localhost:61613"></transportConnector>
                          <transportConnector name="xmpp"        uri="xmpp://localhost:61222"></transportConnector>
                     -->
                </transportConnectors>
      
                <!-- The store and forward broker networks ActiveMQ will listen     to -->
                <networkConnectors>
                     <!-- by     default     just auto discover the other brokers -->
                     <!--
                          <networkConnector name="host1 and host2" uri="static://(tcp://host1:61616,tcp://host2:61616)"/>
                          <networkConnector name="default-nc" uri="multicast://default"></networkConnector>
                     -->
                </networkConnectors>
      
                <!-- Use the following if you wish to configure     the journal with JDBC -->
                <!--
                     <persistenceAdapter>
                     <journaledJDBC journalLogFiles="5" dataDirectory="${activemq.base}/activemq-data"  dataSource="#postgres-ds"></journaledJDBC>
                     </persistenceAdapter>
                -->
      
                <!-- Or     if you want to use pure     JDBC without a journal -->
                <!--
                     <persistenceAdapter>
                     <jdbcPersistenceAdapter     dataSource="#postgres-ds"></jdbcPersistenceAdapter>
                     </persistenceAdapter>
                -->
      
                <!--  Use the following     to set the broker memory limit
                -->
                <persistenceAdapter>
                     <amqPersistenceAdapter maxFileLength="7mb">
                          <usageManager>
                               <systemUsage>
                                    <memoryUsage>
                                         <memoryUsage limit="10 mb"
                                              percentUsageMinDelta="20"></memoryUsage>
                                    </memoryUsage>
                                    <tempUsage>
                                         <tempUsage limit="100 mb"></tempUsage>
                                    </tempUsage>
                                    <storeUsage>
                                         <storeUsage limit="15 mb" name="foo2"></storeUsage>
                                    </storeUsage>
                               </systemUsage>
                          </usageManager>
      
                     </amqPersistenceAdapter>
                </persistenceAdapter>
      
      
                <producerSystemUsage>
                     <systemUsage>
                          <memoryUsage>
                               <memoryUsage limit="10 mb"
                                    percentUsageMinDelta="20"></memoryUsage>
                          </memoryUsage>
                          <tempUsage>
                               <tempUsage limit="100 mb"></tempUsage>
                          </tempUsage>
                          <storeUsage>
                               <storeUsage limit="15 mb" name="foo1"></storeUsage>
                          </storeUsage>
                     </systemUsage>
                </producerSystemUsage>
      
                <!-- Use the following to configure how     ActiveMQ is exposed in JMX
                     <managementContext>
                     <managementContext connectorPort="1099"     jmxDomainName="org.apache.activemq"></managementContext>
                     </managementContext>
                -->
      
           </broker>
      
           <camelContext id="camel"
                xmlns="http://activemq.apache.org/camel/schema/spring">
      
                <!-- You can use a <package> element for each root package to search for Java routes -->
                <package>org.foo.bar</package>
      
                <!-- You can use Spring     XML syntax to define the routes     here using the <route> element -->
                <route>
                     <from uri="activemq:example.A"></from>
                     <to uri="activemq:example.B"></to>
                </route>
           </camelContext>
      
      
      
           <!-- lets create a command agent to respond to message based admin commands on the ActiveMQ.Agent topic     -->
           <commandAgent xmlns="http://activemq.org/config/1.0"></commandAgent>
      
      
           <!-- An     embedded servlet engine     for serving up the Admin console -->
           <jetty xmlns="http://mortbay.com/schemas/jetty/1.0">
                <connectors>
                     <nioConnector port="8161"></nioConnector>
                </connectors>
      
                <handlers>
                     <webAppContext contextPath="/admin"
                          resourceBase="${activemq.base}/webapps/admin" logUrlOnStart="true"></webAppContext>
                     <webAppContext contextPath="/demo"
                          resourceBase="${activemq.base}/webapps/demo" logUrlOnStart="true"></webAppContext>
                </handlers>
           </jetty>
      
           <!--  This xbean configuration file supports all the standard spring xml configuration options -->
      
           <!-- Postgres DataSource Sample     Setup -->
           <!--
                <bean id="postgres-ds" class="org.postgresql.ds.PGPoolingDataSource">
                <property name="serverName" value="localhost"></property>
                <property name="databaseName" value="activemq"></property>
                <property name="portNumber" value="0"></property>
                <property name="user" value="activemq"></property>
                <property name="password" value="activemq"></property>
                <property name="dataSourceName"     value="postgres"></property>
                <property name="initialConnections" value="1"></property>
                <property name="maxConnections"     value="10"></property>
                </bean>
           -->
      
           <!-- MySql DataSource Sample Setup -->
           <!--
                <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
                <property name="driverClassName" value="com.mysql.jdbc.Driver"></property>
                <property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"></property>
                <property name="username" value="activemq"></property>
                <property name="password" value="activemq"></property>
                <property name="poolPreparedStatements"     value="true"></property>
                </bean>
           -->
      
           <!-- Oracle DataSource Sample Setup -->
           <!--
                <bean id="oracle-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
                <property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"></property>
                <property name="url" value="jdbc:oracle:thin:@localhost:1521:AMQDB"></property>
                <property name="username" value="scott"></property>
                <property name="password" value="tiger"></property>
                <property name="poolPreparedStatements"     value="true"></property>
                </bean>
           -->
      
           <!-- Embedded Derby DataSource Sample Setup -->
           <!--
                <bean id="derby-ds" class="org.apache.derby.jdbc.EmbeddedDataSource">
                <property name="databaseName" value="derbydb"></property>
                <property name="createDatabase"     value="create"></property>
                </bean>
           -->
      
      </beans>
      

       

      Edited by: tamirw on Feb 7, 2008 9:19 AM

        • 1. Re: limiting resouces usage in FUSE MB 5.0 (ActiveMQ 5.0)
          rdavies

          Hi Tamir,

           

          I can't read the configuration file - can you attach it again ? I'm assuming you're using the activemq xml  configuration .

          There is supposed to be control on the amount of disk space available for amqstore - but as yet there is no mechanism for reclaiming disk space - but we could provide this as a future enhancement

           

          thanks,

           

          Rob Davies

          • 2. Re: limiting resouces usage in FUSE MB 5.0 (ActiveMQ 5.0)
            tamirw

            Hi Rob,

             

            I tried fixing it, but editing XML here is a real nightmare ...

            That's the best I could do.

            We should be able to attach external files.

             

            Anyway, you're right, I am using the xml configuration, even though I don't mind using the programmatic approach if I need to.

             

            What did you mean by "there is no mechanism for reclaiming disk space" ? If a message expires, doesn't it get removed from the physical store?

             

            Tamir

            • 3. Re: limiting resouces usage in FUSE MB 5.0 (ActiveMQ 5.0)
              rdavies

              Hi Tamir,

               

              I didn't see (though it could be the xml editing - thx for doing that!) - that memoryUsage was a sub element of SystemUsage -

              e.g.

               

              <!--  Use the following to set the broker memory limit
                   <systemUsage>
                               <systemUsage>
                                    <memoryUsage>
                                             <memoryUsage limit="10 mb" percentUsageMinDelta="20"></memoryUsage>
                                    </memoryUsage>
                                    <tempUsage>
                                             <tempUsage limit="100 mb"></tempUsage>
                                    </tempUsage>
                                    <storeUsage>
                                             <storeUsage limit="1 gb" name="foo"></storeUsage>
                                    </storeUsage>
                               </systemUsage>
                       </systemUsage>
                 -->
              

               

              the '' markup is pretty hand btw

               

              Currently messages are only expired when paged into the message broker for delivery - there is an outstanding  issue for this.

               

              cheers,

               

              Rob

              • 4. Re: limiting resouces usage in FUSE MB 5.0 (ActiveMQ 5.0)
                tamirw

                I tried that setting (simply uncommenting the systemUsage tag) but I didn't see any effect to both RAM and disk usage.

                Expired Messages not being reclaimed is indeed a bug, but assuming I can limit the size of the queue (by discarding old messages) it won't be that bad ...

                 

                Have you ever actually tested this configuration and saw that the store size is being bound by those values ?

                What happens when that value is reached?

                • 5. Re: limiting resouces usage in FUSE MB 5.0 (ActiveMQ 5.0)
                  tamirw

                  I'm really stuck on this one, and it's a real bummer, caues from all other aspects, ActiveMQ sounds like the perfect solution for us ...

                   

                  I've started diving into the code, but I'm not sure where to start : should I look at the ActiveMQQueue (since I want to limit the depth of the queue) or in the AMQPersistenceAdapter (since I want to control the disk usage).

                  What do you think ?