[andy.miller]

One final question. If I remove the @CacheConfig annotation, can I add the maxSize and the other parameters to the ejb3-interceptors-aop.xml, like it has for the other two entries?

Scratch that. I already know the answer. I'm just a little tired. It's already in there.

[andy.miller]

"ALRubinger" wrote:

I can probably reduce the code down to just the servlet and stateful session bean, and strip out the database access, and just put a sleep in the method implementation.

In the end we're going to need a more isolated case that may involve invocations on an SFSB alone, or preferably something to test Cache passivation directly. Then we may apply the same test to any of our cache implementations.

S,
ALR

I will work on cutting out just the code for the Stateful Session Bean, so it can be turned into an isolated test case tomorrow.

I have been struggling to create a more isolated test case that's easy to run. I have a test case that is creating lots of Stateful Session Beans, but I can't get them to be cached (other than one bean at a time). What causes the bean to be cached, and then eventually passivated?

[andy.miller]

"ALRubinger" wrote:

I can probably reduce the code down to just the servlet and stateful session bean, and strip out the database access, and just put a sleep in the method implementation.

In the end we're going to need a more isolated case that may involve invocations on an SFSB alone, or preferably something to test Cache passivation directly. Then we may apply the same test to any of our cache implementations.

S,
ALR

I will work on cutting out just the code for the Stateful Session Bean, so it can be turned into an isolated test case tomorrow.

I have been struggling to create a more isolated test case that's easy to run. I have a test case that is creating lots of Stateful Session Beans, but I can't get them to be cached (other than one bean at a time). What causes the bean to be cached, and then eventually passivated?

Okay, never mind. I had a little bug in the stateful session bean, and JBDS was not updating the ear dynamically (as it should have).

I now have an isolated test case that caches the stateful session beans. I will attach the code, and I profile of it soon.

[brian.stansberry]

Great, thanks. I've been diverted into a bunch of areas yesterday and this morning, but will be able to focus on this this afternoon.

[alrubinger]

Great, thanks. I've been diverted into a bunch of areas yesterday and this morning, but will be able to focus on this this afternoon.

Which parts are you going to be looking at, Brian? If we find something with the SimpleStatefulCache outside of Clustering then I don't want to pull you away. :)

S,
ALR

[brian.stansberry]

I want to see why it looks like he's ending up w/ a maxSize of 10,000 rather than 100,000. I've created a trivial SFSB deployment that will match what Andy's got; was planning to look a bit at what's happening to the deployment in a debugger. If the metadata StatefulTreeCache gets says 100,000 for sure it's my problem; if it says 10,000 we can flip a coin for the honors :-)

[brian.stansberry]

Just an update.

First, I deployed a bean with configuration like what you reported Andy, and when StatefulTreeCache was accessing the bean metadata to determine how to set up JBoss Cache's passivation, it got a maxSize=100000. So, that's all working fine.

I deployed the bean along with a little service that creates a configurable # of threads that loop a configurable number of times instantiating new beans. Doesn't use the beans after looking up the proxy and invoking a single getter. Fired it off telling it to use 4 threads to create 200K beans. Am now watching the JMX console view of the bean container to see what happens.

1) The CurrentSize attribute steadily rose past 10K; no apparent limit; eventually got up to over 24K

2) After 5 mins, consistent with the default @CacheConfig.idleTimeoutSeconds=300 in ejb3-interceptors-aop.xml, passivation kicked in and the CurrentSize started dropping a bit while PassivatedCount steadily rose. It would appear the single background thread is able to passivate beans faster than the 4 threads create them.

Judging from some periodic logging I added, it's taking about 6 secs to create 500 beans. The beans aren't particularly heavyweight, the client is in-vm, and there's only one node in the cluster, so no replication going on. So not particularly fast.

But, the passivation part seems to be working as I'd expect.

[alrubinger]

Andy has unintentionally uncovered another issue.

While using the @Clustered in the wrong package he was getting SimpleStatefulCache. I've confirmed my remarks that passivation will prevent new sessions and invocations via some Unit Tests attached to:

https://jira.jboss.org/jira/browse/EJBTHREE-1549

I want to put some more work into those tests for the reasons noted in the ticket before applying them to trunk, and then we'll go for a fix.

S,
ALR

[alrubinger]

"ALRubinger" wrote:
https://jira.jboss.org/jira/browse/EJBTHREE-1549

I've committed tests and a fix for this issue.

r80096 will show that instead of holding the cacheMap's intrinsic lock throughout passivation, we now just make a copy of the Map's contents, then free the lock before running passivation.

Side effect: Perhaps removal task might sneak in there and remove an instance before passivation can come along.

S,
ALR

[wolfc]

There should be no side effects. :-)

Another side effect is that the bean isn't really passivated. :-)

[alrubinger]

"wolfc" wrote:
Another side effect is that the bean isn't really passivated. :-)

Elaborate?

S,
ALR

[andy.miller]

Just an update.

First, I deployed a bean with configuration like what you reported Andy, and when StatefulTreeCache was accessing the bean metadata to determine how to set up JBoss Cache's passivation, it got a maxSize=100000. So, that's all working fine.

I deployed the bean along with a little service that creates a configurable # of threads that loop a configurable number of times instantiating new beans. Doesn't use the beans after looking up the proxy and invoking a single getter. Fired it off telling it to use 4 threads to create 200K beans. Am now watching the JMX console view of the bean container to see what happens.

1) The CurrentSize attribute steadily rose past 10K; no apparent limit; eventually got up to over 24K

2) After 5 mins, consistent with the default @CacheConfig.idleTimeoutSeconds=300 in ejb3-interceptors-aop.xml, passivation kicked in and the CurrentSize started dropping a bit while PassivatedCount steadily rose. It would appear the single background thread is able to passivate beans faster than the 4 threads create them.

Judging from some periodic logging I added, it's taking about 6 secs to create 500 beans. The beans aren't particularly heavyweight, the client is in-vm, and there's only one node in the cluster, so no replication going on. So not particularly fast.

But, the passivation part seems to be working as I'd expect.

In my test, what I see in the JMX console is a -1 in the maxSize. I even have the maxSize changed in ejb3-interceptors-aop.xml to 125,000, so I can tell the difference between the two caches. I have even tried putting the @CacheConfig annotation in, and it still shows -1 in the maxSize in the JMX console.

Is it possible that what you are testing is not what is in the CR 2 build?

[wolfc]

"ALRubinger" wrote:
Elaborate?

S,
ALR

I 'fixed' your unit test. No need to thank me. :-P

[brian.stansberry]

Is it possible that what you are testing is not what is in the CR 2 build?

Oh, for sure. I'm testing my local checkout of trunk, which is probably off from head by a day or so.

Hmm, I see what you are saying about the maxSize in the JMX console. I'll have a look; that's a bug. When I tracked the deployment in the debugger, the 100K maxSize came through from the metadata and was passed into the JBC eviction configuration. So 99% sure the bug is in the JMX display of the metadata.

[andy.miller]

Is it possible that what you are testing is not what is in the CR 2 build?

Oh, for sure. I'm testing my local checkout of trunk, which is probably off from head by a day or so.

Hmm, I see what you are saying about the maxSize in the JMX console. I'll have a look; that's a bug. When I tracked the deployment in the debugger, the 100K maxSize came through from the metadata and was passed into the JBC eviction configuration. So 99% sure the bug is in the JMX display of the metadata.

Based on your test, that's what I was starting to think as well. So, that leaves me with one other issue to explore. Why is my test so much slower using the StatefulTreeCache, then using the SimpleStatefulCache?

I have been having trouble with getting JBoss Profiler working, but Jesper has given me a fix for an issue I was having. Could you recommend specific packages I should make sure I capture with the profiler?