Synchronized clustered timers

[julia.zashchitina]

Hello, everyone.

We run our application on Wildfly 8.2.0.Final in domain mode with ha profile. Database: MySQL 5.6. OS: Ubuntu. Hosting: Amazon AWS.

We have two timers and we need both of them to work only on one node at a time. We also need high availability support for failure cases.

We tried to use two different approaches:

1) Synchronize timers by database as specified by:

https://docs.jboss.org/author/display/WFLY9/EJB3+Clustered+Database+Timers

It only works for Wildfly9+MySQL or Wildfly8+PostgreSQL which is not an option in our case (Wildfly8+MySQL).

2) Synchronize timers using SingletonService, described here: https://github.com/wildfly/quickstart/tree/master/cluster-ha-singleton.

It works perfectly when we test it in our local network or if we deploy only one timer service. However, when we try to deploy application with two timer services on Amazon, deployment hangs (without any errors) and as well as the whole Wildfly server.

Is there any way to run multiple scheduled services on the AWS synchronized via SingletonService or by the database in our configuration?

[wdfink]

1)

If the database option is available you might add the correct DB settings to the module, there is a file for the related timer SQL.

2)

not sure why it hangs. Could you provide a bit more information?

[julia.zashchitina]

1) For synchronizing timers using database we did the following steps:

    a. Modified file WILDFL_HOME}\modules\system\layers\base\org\jboss\as\ejb3\main\timers\timer-sql.properties as the one attached.

    b. Registered datasource and driver in domain.xml in ha profile:

<datasource jta="true" jndi-name="java:/jdbc/timerdb" pool-name="timerDB" enabled="true" use-ccm="true">
                        <connection-url>jdbc:mysql://URL/timerdb</connection-url>
                        <driver>mysql</driver>
                        <transaction-isolation>TRANSACTION_SERIALIZABLE</transaction-isolation>
                        <security>
                                <user-name>username</user-name>
                              <password>password</password>
                        </security>
                        <validation>
                            <validate-on-match>false</validate-on-match>
                            <background-validation>false</background-validation>
                        </validation>
                        <timeout>
                            <set-tx-query-timeout>false</set-tx-query-timeout>
                            <blocking-timeout-millis>0</blocking-timeout-millis>
                            <idle-timeout-minutes>0</idle-timeout-minutes>
                            <query-timeout>0</query-timeout>
                            <use-try-lock>0</use-try-lock>
                            <allocation-retry>0</allocation-retry>
                            <allocation-retry-wait-millis>0</allocation-retry-wait-millis>
                        </timeout>
                        <statement>
                            <share-prepared-statements>false</share-prepared-statements>
                        </statement>
               </datasource>

      c. Changed timer-service:

         <timer-service thread-pool-name="default" default-data-store="clustered-store">
                    <data-stores>
                        <database-data-store name="clustered-store" datasource-jndi-name="java:/jdbc/timerdb" database="mysql" partition="timer"/>
                    </data-stores>
                </timer-service>

       Timer beans are created in our java application with @Singleton annotation and timeout method has @Schedule(second="*/15", minute="*", hour="*", persistent = true) annotation.

    However, with that configuration in Wildfly 8.2.0.Final timer services are not synchronized and timer task is executed on each node (though we do have a record for each timer service in database).  MySQL server version is 5.6, jdbc connector - 5.1.31.

2) The only thing we changed in cluster-ha-singleton source from the above link is quorum argument for install(HATimerService.QUORUM_SERVICE_NAME, 1, context); line in HATimerServiceActivator class. Then we deploy the application and you can see the result in attached log file. After that server hangs, we are not able to remove the deployment or even to stop the server with Ctrl+C.