WildFly 11: stopping the ManagedExecutorService during server stop
hostalp May 4, 2018 11:12 AMHello,
we're testing one application in an earlier development state at WildFly 11 and have hit the following issue:
The application makes use of Spring's DefaultMessageListenerContainer for receiving some messages. The related message listener threads utilize the ManagedExecutorService. This works as expected under normal circumstances.
When we try to stop the server in domain mode (e.g. server-group:stop or server-config:stop-server commands) the server will attempt to enter the suspended state (serverState=running, suspendState=SUSPENDING) and will be stuck in this state unable to proceed further and fully stop.
We identified that this is due to those message listeners which keep running even in the suspended state with no problem.
Based on the JSR-236 spec - The Java Community Process(SM) Program - communityprocess - final
3.1.6.1 Java EE Product Provider Requirements
This subsection describes additional requirements for ManagedExecutorService providers.
1. All tasks, when executed from the ManagedExecutorService, will run with the Java EE component
identity of the component that submitted the task.
2. The lifecycle of a ManagedExecutorService is managed by an application server. All lifecycle operations
on the ManagedExecutorService interface will throw a java.lang.IllegalStateException exception.
This includes the following methods that are defined in the java.util.concurrent.ExecutorService
interface: awaitTermination(), isShutdown(), isTerminated(), shutdown(), and shutdownNow().
Final Release 3-11
3. No task submitted to an executor can run if task’s component is not started.
When a ManagedExecutorService instance is being shutdown by the Java EE Product Provider:
1. All attempts to submit new tasks are rejected.
2. All submitted tasks are cancelled if not running.
3. All running task threads are interrupted.
4. All registered ManagedTaskListeners are invoked.
E.g. the container should reject new tasks, cancel those already submitted, interrupt the running ones etc.
Sort of this can be seen with the ManagedScheduledExecutorService where in this state we get:
ERROR [org.jboss.as.ee] (EE-ManagedScheduledExecutorService-default-Thread-4) WFLYEE0110: Failed to run scheduled task: java.lang.IllegalStateException: WFLYEE0111: Cannot run scheduled task javax.enterprise.concurrent.ManagedExecutors$ManagedRunnable@11f8633a as container is suspended
at org.jboss.as.ee.concurrent.ControlPointUtils$ControlledScheduledRunnable.run(ControlPointUtils.java:164)
at org.jboss.as.ee.concurrent.ControlPointUtils$ControlledManagedRunnable.run(ControlPointUtils.java:246)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.glassfish.enterprise.concurrent.internal.ManagedFutureTask.run(ManagedFutureTask.java:141)
at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$101(ManagedScheduledThreadPoolExecutor.java:383)
at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:532)
at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedTriggerSingleFutureTask.run(ManagedScheduledThreadPoolExecutor.java:587)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250)
at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78)
However tasks that are submitted to the ManagedExecutorService are still seen to be successfully accepted and processed, none are interrupted etc. - e.g. they just keep running, preventing the suspend operation from completing.
Is this really correct behavior? Then how is one supposed to detect this state in order to stop the related mesage listeners?
Note shutdown or kill commands are able to stop the server, but not the stop or server-stop.