-
1. Re: EJB3 / $Proxy110465 / jboss RemotingClassLoader
snacker Jun 19, 2010 12:40 AM (in response to snacker)However, in addition to the above it says it is running out of memory, but the heap and permgen look fine to me (still 1000's of proxys though):
This is what jmap shows:
bash-2.05# jmap -heap 14912
Attaching to process ID 14912, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 14.0-b16using thread-local object allocation.
Parallel GC with 2 thread(s)Heap Configuration:
MinHeapFreeRatio = 40
MaxHeapFreeRatio = 70
MaxHeapSize = 1073741824 (1024.0MB)
NewSize = 2228224 (2.125MB)
MaxNewSize = 4294901760 (4095.9375MB)
OldSize = 4194304 (4.0MB)
NewRatio = 2
SurvivorRatio = 8
PermSize = 16777216 (16.0MB)
MaxPermSize = 268435456 (256.0MB)Heap Usage:
PS Young Generation
Eden Space:
capacity = 316014592 (301.375MB)
used = 7066480 (6.7391204833984375MB)
free = 308948112 (294.63587951660156MB)
2.2361245900948776% used
From Space:
capacity = 8323072 (7.9375MB)
used = 0 (0.0MB)
free = 8323072 (7.9375MB)
0.0% used
To Space:
capacity = 23920640 (22.8125MB)
used = 0 (0.0MB)
free = 23920640 (22.8125MB)
0.0% used
PS Old Generation
capacity = 239075328 (228.0MB)
used = 91060992 (86.842529296875MB)
free = 148014336 (141.157470703125MB)
38.08882863898026% used
PS Perm Generation
capacity = 109051904 (104.0MB)
used = 78839208 (75.18692779541016MB)
free = 30212696 (28.813072204589844MB)
72.29512288020207% usedAll of the OutOfMemoryErrors occur in java.lang.Class.getDeclaredFields0(Native Method) as follows:
java.lang.OutOfMemoryError
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2291)
at java.lang.Class.getDeclaredField(Class.java:1880)
at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1610)
at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:52)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:425)
at java.security.AccessController.doPrivileged(Native Method)
at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310)
at java.io.ObjectStreamClass.initProxy(ObjectStreamClass.java:507)
at java.io.ObjectInputStream.readProxyDesc(ObjectInputStream.java:1539)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1493)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at org.jboss.aop.joinpoint.InvocationResponse.readExternal(InvocationResponse.java:119)
at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1792)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1751)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
at org.jboss.remoting.serialization.impl.java.JavaSerializationManager.receiveObjectVersion2_2(JavaSerializationManager.java:238)
at org.jboss.remoting.serialization.impl.java.JavaSerializationManager.receiveObject(JavaSerializationManager.java:138)
at org.jboss.remoting.marshal.serializable.SerializableUnMarshaller.read(SerializableUnMarshaller.java:123)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedRead(MicroSocketClientInvoker.java:1299)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:907)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:167)
at org.jboss.remoting.Client.invoke(Client.java:1927)
at org.jboss.remoting.Client.invoke(Client.java:770)
at org.jboss.aspects.remoting.InvokeRemoteInterceptor.invoke(InvokeRemoteInterceptor.java:60)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.ejb3.proxy.impl.remoting.IsLocalProxyFactoryInterceptor.invoke(IsLocalProxyFactoryInterceptor.java:72)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
at $Proxy2.createProxyBusiness(Unknown Source)
at org.jboss.ejb3.proxy.impl.objectfactory.session.SessionProxyObjectFactory.createProxy(SessionProxyObjectFactory.java:129)
at org.jboss.ejb3.proxy.impl.objectfactory.session.stateless.StatelessSessionProxyObjectFactory.getProxy(StatelessSessionProxyObjectFactory.java:79)
at org.jboss.ejb3.proxy.impl.objectfactory.ProxyObjectFactory.getObjectInstance(ProxyObjectFactory.java:158)
at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:304)
at org.jnp.interfaces.NamingContext.getObjectInstance(NamingContext.java:1479)
at org.jnp.interfaces.NamingContext.getObjectInstanceWrapFailure(NamingContext.java:1496)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:822)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:686)
at javax.naming.InitialContext.lookup(InitialContext.java:392)... our other business methods below ...
Is it possible the native code is getting a different error, but just interpreting it as an OutOfMemoryError?
-
2. Re: EJB3 / $Proxy110465 / jboss RemotingClassLoader
jaikiran Jun 28, 2010 7:08 AM (in response to snacker)Nathan Ciliberto wrote:
From the heap dump the number starts $Proxy108810 and ends $Proxy110465, so over 10000 of these are hanging around.
There are 6253 of the org.jboss.remoting.loading.RemotingClassLoader instances in the heap as well... why?
Is there any way to determine why so many of these (proxy's and RemotingClassLoaders) are being created?
How are you using those proxies in your application? Can you provide more details about your application? Are you caching those proxies? How do you do the lookup?
-
3. Re: EJB3 / $Proxy110465 / jboss RemotingClassLoader
snacker Jun 29, 2010 11:42 PM (in response to jaikiran)1 of 1 people found this helpfulWe are doing the ejb3 lookups as follows:
Hashtable ht = new Hashtable();
InitialContext ic;
Object ejbObj;
MyRemoteEjbInterface ejb;ht.put( "java.naming.provider.url" , "jnp://serverA:1099,jnp://serverB:1099,jnp://serverC:1099" );
ht.put( "java.naming.factory.url.pkgs" , "org.jboss.naming:org.jnp.interfaces" );
ht.put( "java.naming.factory.initial" , "org.jnp.interfaces.NamingContextFactory" );
ht.put( "jnp.disableDiscovery" , "true" );
ht.put( "jnp.sotimeout" , "100" );
ht.put( "jnp.timeout" , "100" );ic = new InitialContext( ht );
ejbObj = ic.lookup( jndiName );
ejb = (MyRemoteEjbInterface) ejbObj;... call methods on ejb ...
For ejb2 references we call PortableRemoteObject.narrow(...) when appropriate, then call the create() method on the reference.
We are not caching the ejb object returned... is it safe to do so? What about if the jboss instance goes down or sharing these between web sessions or other threads?
-Nathan
-
4. Re: EJB3 / $Proxy110465 / jboss RemotingClassLoader
snacker Nov 12, 2010 5:04 PM (in response to snacker)1 of 1 people found this helpfulWe have an update on this issue.
We have had some free time and decided to take a look at this again.
We created a simple test class that just loops forever calling "ic.lookup( jndiName )" until an error occurs.
It usually errors after ~7000 calls on certain EJB's.
So far it appears to be due to a bug in jdk 1.6.0_20.
We have tried it with jdk 1.6.0_22 and have not been able to reproduce it.
We have not tested this with jdk 1.6.0_21.
Also, we are not sure what was specifically fixed since *_20 that may have fixed this.
-
5. Re: EJB3 / $Proxy110465 / jboss RemotingClassLoader
wdfink Nov 13, 2010 6:06 AM (in response to snacker)We are not caching the ejb object returned... is it safe to do so? What about if the jboss instance goes down or sharing these between web sessions or other threads?
The references (in your case MyRemoteEjbInterface ejb) can be 'cached' and reused if it is a reference to a SLSB!
In case of crash/shutdown of one node of a cluster (if you have one) the proxy is updated by next access with the actual clusterview.
In case of all nodes (or the only one) are down I suppose it depends to the configuration, for EJB2 there is a special client-side-interceptor whithin the chain to handle this 'reconnect' (AFAIK it is not the standard configuration)
For EJB3 I'm not sure about this I've it not tested now.