-
1. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
jaikiran Aug 21, 2014 2:24 AM (in response to abl-dam)Can you attach the server.log which shows this 2 minute deployment logs? Also, have you been able to profile (or at least take thread dumps at regular intervals) during this 2 minute long deployment? Do you have any jboss-ejb-client.xml in your deployment, if yes, what does it look like?
-
2. Re: Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
abl-dam Aug 21, 2014 3:56 AM (in response to jaikiran)thanks for your reply!
attached is the server.log from startup, using the out-of-the-box log-config. if you need one with more logging details, please let me know
also I took some thread dumps during the startup, and one shortly after
no, no jboss-ejb-client.xml.
a jboss-deplyment-structure.xml is the only jboss specific DD
-
server.zip 9.0 KB
-
threaddump.zip 78.4 KB
-
-
3. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
jblinick Aug 22, 2014 4:24 PM (in response to abl-dam)we are just starting out with jbpm, but can confirm that it seems very slow ( >1.5 mins) in Wildfly (haven't tried AS7).
-
4. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
ctomc Aug 22, 2014 5:22 PM (in response to jblinick)Only thing that kinda stands out from your thread dumps is that there is some remote RMI calls going on which could be a cause of the slowness.
Is you app doing some RMI lookups during deployment? That could make it slow, especially if remote host to which RMI connection is done is slow to respond.
That would also explain this entry in server.log
2014-08-21 09:43:54,318 ERROR [org.jboss.remoting.remote.connection] (XNIO-1 I/O-1) JBREM000200: Remote connection failed: java.io.IOException: Eine bestehende Verbindung wurde softwaregesteuert durch den Hostcomputer abgebrochen
-
5. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
abl-dam Aug 25, 2014 3:47 AM (in response to ctomc)@Tomaz, Josh ... thanks for your replies
no, we make no RMI lookups at all in out application (at least not explicit in our code). maybe something in our code is causing RMI lookups ... can you see the origin for this lookups in the threaddump?
as stated in the original post: the identical application deploys fast in AS7/EAP6 - without any errors on deployment
-
6. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
abl-dam Oct 10, 2014 7:57 AM (in response to abl-dam)I came back to this and did some more tests. I tracked it down by comparing logs and putting this in log config:
<logger category="org.jboss.weld"> <level name="TRACE"/> </logger>
the time difference results mostly from WELD deployment:
WILDFLY 8.1.0
13:08:23,603 TRACE [org.jboss.weld.Bean] (weld-worker-1) WELD-001541: Adding method to proxy: ... (first entry)
...
13:09:28,192 TRACE [org.jboss.weld.Bean] (weld-worker-4) WELD-001541: Adding method to proxy: ... (last entry)
-> 64,5 sec.JBoss EAP 6.0.1
13:30:40,049 TRACE [org.jboss.weld.Bean] (MSC service) Adding method ... (first entry)
...
13:30:44,629 TRACE [org.jboss.weld.Bean] (MSC service) Adding method ... (last entry)
-> 4,5 sec.although WFLY starts 4 worker threads for WELD scanning and deployment (weld-worker-1 to 4), it needs about 1 min. longer for the job.
Any ideas how to avoid or improve that?
We are still not able to move on to Wildfly (or Java8) and stick with old JB7/EAP6 Version!
-
7. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
ctomc Oct 13, 2014 6:41 AM (in response to abl-dam)1 of 1 people found this helpfulAny chance you can try with 9.0.0.Alpha1 which has new Weld 2.2 in it which has lots of fixes for boot time performance.
Same weld is also in 8.x branch and will be part of 8.2 release we are working on.
9.0.0.Alpha1 shouldn't be much different when it comes to deployment then 8.x stuff, just build / assembly / code organization is what changed the most.
-
8. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
abl-dam Oct 10, 2014 10:12 AM (in response to ctomc)ok, I will give it a try. I'll give you feedback asap
do you know when beta 1 comes out? (should be today according to wlfy roadmap)
-
9. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
abl-dam Oct 13, 2014 6:35 AM (in response to ctomc)MUCH better with 9.0.0.Alpha1 !
Startup / deployment in about 28,5 sec. (comparing to 120 sec. in 8.1.0.Final):
12:20:37,789 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 9.0.0.Alpha1 (WildFly Core 1.0.0.Alpha8) started in 28482ms - Started 1141 of 1244 services (176 services are lazy, passive or on-demand)
12:20:28,468 TRACE [org.jboss.weld.Bean] (MSC service thread 1-5) WELD-001541: Adding method to Proxy ... (first entry)
...
12:20:37,384 TRACE [org.jboss.weld.Bean] (MSC service thread 1-3) WELD-001541: Adding method to Proxy ... (last entry)
-> 9 sec.
but still about double time compared to EAP6
the new version also writes out the following lines:
12:20:26,017 TRACE [org.jboss.weld.deployer] (weld-worker-1) Falling back to reflection for org.apache.wicket.markup.html.form.Form
...
12:20:28,312 TRACE [org.jboss.weld.deployer] (weld-worker-4) Falling back to reflection for org.apache.wicket.markup.html.panel.Panel
...
nearly all for wicket classes. we use wicket and have it's libs in the EAR, but also a lot of other libraries. any idea why?
again, it makes no difference if I have this in jboss-all.xml. not in matters of startup time or trace output:
<jboss xmlns="urn:jboss:1.0">
<weld xmlns="urn:jboss:weld:1.0" require-bean-descriptor="true"/>
</jboss>
-
10. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
ctomc Oct 13, 2014 6:41 AM (in response to abl-dam)maybe jharting can help.
-
11. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
jharting Oct 17, 2014 11:58 AM (in response to abl-dam)abl-dam glad to hear you observe improvement with Weld 2.2 (WildFly 9). Until WildFly 8.2 is released, you can use Weld 2.2 with WildFly 8.1.0.Final by applying a patch that can be found here: http://sourceforge.net/projects/jboss/files/Weld/2.2.5.Final/
Anyway, thanks for attaching stack dumps. These are very useful for us. Any chance you could do the same with WildFly 9 Alpha for us (or 8.1.0 with Weld 2.2)? You can also add a text file named WEB-INF/classes/org.jboss.weld.executor.properties
containing:
threadPoolDebug=true
to your application. This will cause more performance-related information to be printed into the server log.
-
12. Re: Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
abl-dam Oct 20, 2014 6:43 AM (in response to jharting)attaching server.log and thread dumps taken from 2 startups for Wildfly9/Weld 2.2
file org.jboss.weld.executor.properties is added to WEB-INF/classes
you can see also the TRACE outputs in there regarding wicket classes, although they should be completely ignored by Weld (IMO)
if I can give you any further information or tracing output, please let me know!
thanks for the provided link to patch Wildfly 8.1 to Weld 2.2 - I will try that also and give you the results
-
13. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
abl-dam Oct 22, 2014 6:56 AM (in response to jharting)applied the Weld 2.2.5 patch above to Wildfly 8.1.0 and tested again
result is as expected: 28-29 sec. for Startup ... same as with Wildfly 9 Alpha
-
14. Re: Deployment and Startup very slow on Wildfly 8.1.0 compared to AS7/EAP6
karthik-vasishta.ramesh Jan 23, 2015 9:32 PM (in response to jharting)Thanks a lot Jozef. I applied the weld 2.2.5 patch that you mentioned on Wildfly8.1.0.Final and the performance is so much better!