-
30. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
rajiv.shivane May 15, 2018 7:50 AM (in response to adinn)adinn wrote:
Apologies for being dense -- too much context-switching.
Happens to me all the time
We are just glad you are able to make time for this. This would be a good improvement for a wide set of byteman users.
adinn wrote:
Ah, sorry, I was indeed referring to the class file byte[] -- under the mistaken belief that the the Checker would need to retain a reference to it in order to recompute the details of the supers and interfaces. Clearly, ByteCodeChecker instances include all the necessary data and there is little or no danger of incurring any significant storage cost by retaining them in a cache.
In fact we could make ByteCodeChecker leaner by copying the four fields from ClassStructureAdapter in the constructor, instead of holding a reference to the CSA and delegating the method calls to it.
-
31. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
adinn May 18, 2018 9:53 AM (in response to rajiv.shivane)Hi Rajiv,
I have raised and fixed BYTEMAN-36, adding a new cache for ClassChecker instances to the master and 3.X branches.
I also added an option to trace cache puts and cache hits, enabling stats to be gathered (set -D org.jboss.byteman.checker.dumpstats=true on the command line to enable tracing). I tested this using Wildlfy 10 with one overriding CLASS rule and one overriding interface rule and it was very successful in reducing lookups. Here are the rules (they are defined never to actually generate a match):
RULE test INTERFACE Foo METHOD foo AT ENTRY IF TRUE DO traceln("Foo.foo()"); ENDRULE
RULE test2 CLASS Bar METHOD bar AT ENTRY IF TRUE DO traceln("Bar.bar()"); ENDRULE
The final counts printed after Widlfly startup are
maps 100 classes 1532 bytes 1792904 hits 3202 hit bytes 1982611
i.e. lookups occurred from 100 different classloaders for 1532 classes (that includes some repeated lookup of the same super/interface from different starting loader). The number of resource bytes loaded was 1792904 and the number of extra bytes that would have been loaded without the cache was 1982611.
I also collected and analysed the names of the classes associated with each Put an Hit listed in the trace output:
Total Puts: 1532 Total Hits: 3202 Unique Puts: 1072 Unique Hits: 425
The first thing to note is that the hit rate is just over 2 Hits per Put which matches the byte load savings.
The ratio of Total Puts to Unique Puts indicates that there are quite a few cases where a super or interface is looked up from subclasses that belong to different loaders (on average 1 out of every 2 classes but actually there is quiet a small number of classes that have many lookups). That figure would have been quite a bit higher without a trick I worked out. Classes whose name starts with "java." can only ever be loaded via the bootstrap loader. So, for that case I always look up the checker or, failing that, the class bytes resource using the bootstrap loader. So, there is only 1 Put and many Hits for java.lang.Serializable, java.lang.Runnable, java.util.Collection etc. Without this trick these classes get anywhere from 15 to 60 puts in as many different loader maps.
The ratio of Unique Hits to Total Hist is quite high. That's doesn't indicate that the cache is not working. On the contrary, it's merely a reflection of the fact that many classes get looked up multiple times from the same class loader as well as occasionally from a different class loader.
Anyway, the acid test is startup times. Without the cache the above two rules added about 1 sec to startup time. With the cache the additional time is around 100 msecs. So, this is indeed a big win, as expected. I hope it
I have deployed a new 4.0.3-SNAPSHOT build to the maven central repo. I hope it helps improve your startup. Let me know if you have any further problems.
regards,
Andrew Dinn
-
32. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
rajiv.shivane May 21, 2018 2:52 AM (in response to adinn)adinn wrote:
I have raised and fixed BYTEMAN-36, adding a new cache for ClassChecker instances to the master and 3.X branches.
Thanks Andrew, this is great news.
I looked at the commit on github. I posted a query about the commit at: add cache for ClassByteCheckers -- fixes BYTEMAN-368 · bytemanproject/byteman@9f650be · GitHub
We will give try the snapshot build and update you. These changes are exactly same as the ones we tested with the customer, so I expect it to perform exactly the same.
Thanks again,
Rajiv
-
33. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
rajiv.shivane May 22, 2018 1:08 AM (in response to adinn)adinn wrote:
I have deployed a new 4.0.3-SNAPSHOT build to the maven central repo. I hope it helps improve your startup. Let me know if you have any further problems.
Hi Andrew,
We have deployed the new version with 4.0.3-SNAPSHOT internally. It has same startup time as the patch we had tried with the customer.
It will take couple of weeks before we can try this version with the customer. Could we go ahead and make the 4.0.3 release without waiting for the results (which I am confident are going to be positive)?
Thanks,
Rajiv
-
34. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
adinn May 22, 2018 4:01 AM (in response to rajiv.shivane)Hi Rajiv,
Thanks for the feedback. I am on holiday for 2 weeks from noon on Thursday. I wlll try to get a new release out before then.
regards,
Andrew Dinn
-
35. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
rajiv.shivane May 22, 2018 4:03 AM (in response to adinn)Thanks Andrew, much appreciated!
-
36. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
rajiv.shivane Jun 12, 2018 5:16 AM (in response to adinn)adinn wrote:
Thanks for the feedback. I am on holiday for 2 weeks from noon on Thursday. I wlll try to get a new release out before then.
Hi Andrew, Hope you had a good holiday. Do you have a time-frame in mind for the new byteman release?
Thanks,
Rajiv
-
37. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
adinn Jul 3, 2018 3:58 AM (in response to rajiv.shivane)Hi Rajiv,
Byteman 4.0.3 and 3.0.13 are now released.
regards,
Andrew Dinn
-
38. Re: Server startup too slow because of potential logic issue in LoadCache.lookupClass
rajiv.shivane Jul 3, 2018 4:08 AM (in response to adinn)adinn wrote:
Hi Rajiv,
Byteman 4.0.3 and 3.0.13 are now released.
regards,
Andrew Dinn
Thanks Andrew, much appreciated!