Can you attach the server.log which shows this 2 minute deployment logs? Also, have you been able to profile (or at least take thread dumps at regular intervals) during this 2 minute long deployment? Do you have any jboss-ejb-client.xml in your deployment, if yes, what does it look like?
thanks for your reply!
attached is the server.log from startup, using the out-of-the-box log-config. if you need one with more logging details, please let me know
also I took some thread dumps during the startup, and one shortly after
no, no jboss-ejb-client.xml.
a jboss-deplyment-structure.xml is the only jboss specific DD
we are just starting out with jbpm, but can confirm that it seems very slow ( >1.5 mins) in Wildfly (haven't tried AS7).
Only thing that kinda stands out from your thread dumps is that there is some remote RMI calls going on which could be a cause of the slowness.
Is you app doing some RMI lookups during deployment? That could make it slow, especially if remote host to which RMI connection is done is slow to respond.
That would also explain this entry in server.log
2014-08-21 09:43:54,318 ERROR [org.jboss.remoting.remote.connection] (XNIO-1 I/O-1) JBREM000200: Remote connection failed: java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert durch den Hostcomputer abgebrochen
@Tomaz, Josh ... thanks for your replies
no, we make no RMI lookups at all in out application (at least not explicit in our code). maybe something in our code is causing RMI lookups ... can you see the origin for this lookups in the threaddump?
as stated in the original post: the identical application deploys fast in AS7/EAP6 - without any errors on deployment
I came back to this and did some more tests. I tracked it down by comparing logs and putting this in log config:
<logger category="org.jboss.weld"> <level name="TRACE"/> </logger>
the time difference results mostly from WELD deployment:
13:08:23,603 TRACE [org.jboss.weld.Bean] (weld-worker-1) WELD-001541: Adding method to proxy: ... (first entry)
13:09:28,192 TRACE [org.jboss.weld.Bean] (weld-worker-4) WELD-001541: Adding method to proxy: ... (last entry)
-> 64,5 sec.
JBoss EAP 6.0.1
13:30:40,049 TRACE [org.jboss.weld.Bean] (MSC service) Adding method ... (first entry)
13:30:44,629 TRACE [org.jboss.weld.Bean] (MSC service) Adding method ... (last entry)
-> 4,5 sec.
although WFLY starts 4 worker threads for WELD scanning and deployment (weld-worker-1 to 4), it needs about 1 min. longer for the job.
Any ideas how to avoid or improve that?
We are still not able to move on to Wildfly (or Java8) and stick with old JB7/EAP6 Version!
1 of 1 people found this helpful
Any chance you can try with 9.0.0.Alpha1 which has new Weld 2.2 in it which has lots of fixes for boot time performance.
Same weld is also in 8.x branch and will be part of 8.2 release we are working on.
9.0.0.Alpha1 shouldn't be much different when it comes to deployment then 8.x stuff, just build / assembly / code organization is what changed the most.
ok, I will give it a try. I'll give you feedback asap
do you know when beta 1 comes out? (should be today according to wlfy roadmap)
MUCH better with 9.0.0.Alpha1 !
Startup / deployment in about 28,5 sec. (comparing to 120 sec. in 8.1.0.Final):
12:20:37,789 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 9.0.0.Alpha1 (WildFly Core 1.0.0.Alpha8) started in 28482ms - Started 1141 of 1244 services (176 services are lazy, passive or on-demand)
12:20:28,468 TRACE [org.jboss.weld.Bean] (MSC service thread 1-5) WELD-001541: Adding method to Proxy ... (first entry)
12:20:37,384 TRACE [org.jboss.weld.Bean] (MSC service thread 1-3) WELD-001541: Adding method to Proxy ... (last entry)
-> 9 sec.
but still about double time compared to EAP6
the new version also writes out the following lines:
12:20:26,017 TRACE [org.jboss.weld.deployer] (weld-worker-1) Falling back to reflection for org.apache.wicket.markup.html.form.Form
12:20:28,312 TRACE [org.jboss.weld.deployer] (weld-worker-4) Falling back to reflection for org.apache.wicket.markup.html.panel.Panel
nearly all for wicket classes. we use wicket and have it's libs in the EAR, but also a lot of other libraries. any idea why?
again, it makes no difference if I have this in jboss-all.xml. not in matters of startup time or trace output:
<weld xmlns="urn:jboss:weld:1.0" require-bean-descriptor="true"/>
abl-dam glad to hear you observe improvement with Weld 2.2 (WildFly 9). Until WildFly 8.2 is released, you can use Weld 2.2 with WildFly 8.1.0.Final by applying a patch that can be found here: http://sourceforge.net/projects/jboss/files/Weld/2.2.5.Final/
Anyway, thanks for attaching stack dumps. These are very useful for us. Any chance you could do the same with WildFly 9 Alpha for us (or 8.1.0 with Weld 2.2)? You can also add a text file named WEB-INF/classes/org.jboss.weld.executor.properties
to your application. This will cause more performance-related information to be printed into the server log.
attaching server.log and thread dumps taken from 2 startups for Wildfly9/Weld 2.2
file org.jboss.weld.executor.properties is added to WEB-INF/classes
you can see also the TRACE outputs in there regarding wicket classes, although they should be completely ignored by Weld (IMO)
if I can give you any further information or tracing output, please let me know!
thanks for the provided link to patch Wildfly 8.1 to Weld 2.2 - I will try that also and give you the results
applied the Weld 2.2.5 patch above to Wildfly 8.1.0 and tested again
result is as expected: 28-29 sec. for Startup ... same as with Wildfly 9 Alpha
Thanks a lot Jozef. I applied the weld 2.2.5 patch that you mentioned on Wildfly8.1.0.Final and the performance is so much better!