JBAS015052 trying to deploy applications on AIX

[jasipher]

I have a couple EAR files that work without problems in JBoss AS 7.1.2.Final on Windows, HP-UX, Solaris, and RHEL 6.3, but fail with "JBAS015052: Did not receive a response to the deployment operation within the allowed timeout period [180 seconds]. Check the server configuration file and the server logs to find more about the status of the deployment" on AIX.

We have the same problem on three different AIX boxes, all of which are at slightly different AIX levels: 7100-00-04-1140, 7100-01-04-1216, and another that I don't have access to but have been told is different from these two. The output from java -version on the two I have access to is

java version "1.6.0"

Java(TM) SE Runtime Environment (build pap6460sr10fp1-20120321_01(SR10 FP1))

IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 AIX ppc64-64 jvmap6460sr10fp1-20120202_101568 (JIT enabled, AOT enabled)

J9VM - 20120202_101568

JIT  - r9_20111107_21307ifx1

GC   - 20120202_AA)

JCL  - 20120320_01

When the container tries to deploy the application, it gets through the stage where it lists the JNDI bindings for the EJBs and then just waits for the deployment-timeout to expire.

This knowledge base article suggests getting a thread dump to verify that there's nothing waiting for a call to System.exit to finish, so I tried that. Bizarrely, when I use 'kill -3 PID' to get the thread dump, the container finishes its initialization and the application comes up and runs just fine. I've done a little bit of experimenting around that, and it looks as if I have to delete myapp.ear.deployed (or myapp.ear.failed) from the deployments folder before starting JBoss in order for that little trick to work. I haven't taken the time to nail that down so that I'm sure it's 100% repeatable because it's not really an acceptable workaround.

Any suggestions will be appreciated.

[jasipher]

I built the jboss-as-ejb-in-ear quickstart project and deployed it to the plain vanilla standalone server. That also fails with "JBAS015052: Did not receive a response to the deployment operation within the allowed timeout period [180 seconds]. Check the server configuration file and the server logs to find more about the status of the deployment" on AIX. The server logs are attached.

[ctomc]

Hi,

can you try with jboss 7.2.0.final (=EAP6.1.Alpha)

as there ware few bugs fixed in networking library that effected AIX.

--

tomaz

[jasipher]

I get the same result with eap-6.1.0.Alpha. The only difference is that the JBAS015052 message is not written to the log. It is written to jboss-as-ejb-in-ear-ear.failed. Both the log and the .failed file are attached.

[lafr]

FYI: according to https://community.jboss.org/message/758708#758708#758708 a downgrade of your JDK from SR10 to SR7 (as far as available on AXI 7.1) helps to some extend, but other problems remain as far as I understood.

[jasipher]

Unfortunately SR7 is  no longer available for download so I don't think that's going to work. And, after reading about the other problems with SR7 I'm not sure that's really a workable solution even if we're able to get a copy of it.

[jasipher]

I can't really pretend that I know what's going on in this code, but it looks to me as if the issue is in this method from the BootTimeScannerDeployment class in org.jboss.as.server.deployment.scanner.DeploymentScannerAdd.

 {code}
       @Override
        public Future<ModelNode> deploy(final ModelNode operation, final ScheduledExecutorService scheduledExecutor) {
            try {
                deploymentOperation.set(operation);
                final FutureTask<ModelNode> task = new FutureTask<ModelNode>(new Callable<ModelNode>() {
                    @Override
                    public ModelNode call() throws Exception {
                        deploymentDoneLatch.await();
                        return deploymentResults.get();
                    }
                });
                scheduledExecutor.submit(task);
                return task;
            } finally {
                scanDoneLatch.countDown();
            }
        }
 {code}

The problem is that all it does is wait for deploymentDoneLatch to be counted down without ever executing the operation. I'm really not clear on how this works on the other platforms, but it seems like this method needs an instance of ModelControllerClient that it can use to execute the operation.

I must be missing something because it obviously works everywhere but AIX. I *think* what's going on is that this gets invoked during the oneOffScan (this is what I see in debug), and it just waits for something that never happens. On the non-AIX platforms, I think a second instance of FileSystemDeploymentService is started by the DeploymentScannerService, and that scanner actually does the deployment and somehow counts down deploymentDoneLatch.

This stuff is a bit over my head, and I'd really appreciate it if someone could straighten me out on how it's supposed to work.

[nickarls]

I don't think that's the actual deployment code, I think it's just code for adding stuff to the deployment queue. Perhaps the scanner collects stuff that needs to be deployed then sorts out the dependencies and finally deploys them in the specific order at the end.

In any case, if you would be correct I don't think anything would deploy on any platform

[jasipher]

The problem is resolved with SR13.