This weekend we have started seeing the following on our wildfly master CI builds:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.5.5:single (assemble-src) on project wildfly-dist: Failed to create assembly: Error creating assembly archive src: Problem creating zip: /home/hudson/workspace/narayana/PROFILE/BLACKTIE/jdk/jdk8.latest/label/linux64el6/jboss-as/dist/../testsuite/integration/clustering/src/test/java/org/jboss/as/test/clustering/cluster/provider/bean/ServiceProviderRetrieverBean.java (Too many open files) -> [Help 1]

We use the default ulimit of 1024 max open files.

Based on our last successful CI run and the first failure we can surmise that the commit that tipped the balance lies between the commits Comparing 1e3a4a3f9b4079df59c0a7e0acb67a25d2797067...749a6dedc64aa61a49ad522f17fa824bcfc31a48 · wildfly/wildfly · GitHub

The most likely culprit was the wildfly core upgrade to Alpah10. The set of diffs in the upgrade is Comparing 2.0.0.Alpha9...2.0.0.Alpha10 · wildfly/wildfly-core · GitHub Maybe there are sockets from the previous module test still lingering in the FIN_WAIT-2 state.

We have worked around the issue by increase the ulimit to 10000 open files.