cacheing homeinterfaces in a clustered environment

[nobel]

running jboss-3.2.3 and a copy of the all config and a remote clustered stateless session bean deployed on two jboss instances.

a client in another jvm does a lookup on the home interface, assuming the home interface isnt cached, and upon successful lookup puts the home proxy in a cache. given that the jboss instance to which the home interface was bound is brought down then a subsequent lookup from the client will hit the "stale home inteface" now in the cache. and to my surprise trying to obtain the remote interface from the "stale home interface" will NOT failover to the other jboss instance. calling the create method on the "stale home interface" will generate a "service unavailable" remote exception.

if i remove the caching mechanism everything works. but why doesnt cacheing of home interfaces work?

cheers niklas

[darranl]

Looks like you have not configured your application correctly, what steps have you performed to set up your beans for the cluster.

[nobel]

all my remote SLSB are configured as shown in the jboss.xml snippet below

<session>
 <ejb-name>ServiceFacadeBean</ejb-name>
 <jndi-name>ejb/ServiceFacadeRemote</jndi-name>
 <clustered>True</clustered>
...

the client sets java.naming.provider.url= and thus relies on automatic discovery. jboss0 uses JNDI/HAJNDI 1199/1200 and jboss1 uses 1299/1300. seems as HAJNDI works with failover but as if my beans are not clustered...

am i missing something?

[nobel]

from the trace below i draw the conclusion that cacheing homeiterfaces (or remote interfaces) in a clustered environment is not a good idea. the JRMPInvokerProxyHA keeps a list of available targets that will be exhausted if you shut down jboss instances in the "right" order. in the example i use 2 jboss instances (jboss0 and jboss1) of which jboss1 has been shut down. i run the client and the JRMPInvokerProxyHA will correctly call the only target in the list == jboss0. shutdown jboss0 and fire up jboss1. run the client again with cached homeinterfaces. the JRMPInvokerProxyHA instance has not been refreshed, as it is cached, and it will try to call the only target in the list == jboss0. ConnectException and there are no other targets in the list to failover to throw a ServiceUnavailable exception. cheers niklas.

//***** jboss0 up, jboss1 down and cache empty
2004-06-24 12:47:54,870 INFO [se.teligent.smp.throttlerule.client.P90ThrottleRuleImpl] Inside P90ThrottleRuleImpl()
2004-06-24 12:49:44,269 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] JNDI lookup java:comp/env/ejb/ServiceFacadeRemote
2004-06-24 12:49:44,299 INFO [se.teligent.tsp.servicelocator.client.ServiceLocator] Creating new instance of:se.teligent.tsp.servicelocator.client.ServiceLocator@f01a1e
2004-06-24 12:49:44,608 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Init, clusterInfo: org.jboss.ha.framework.interfaces.FamilyClusterInfoImpl@5e3bd371{familyName=DefaultPartition/jboss.j2ee:jndiName=ejb/ServiceFacadeRemote,service=EJB/H,targets=[org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]],currentViewId=-238527116,isViewMembersInSyncWithViewId=false,cursor=999999999,arbitraryObject=null}, policy=org.jboss.ha.framework.interfaces.RoundRobin@7109c4
2004-06-24 12:49:44,629 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] JNDI lookup java:comp/env/ejb/ThrottleRuleFacade
2004-06-24 12:49:44,763 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Init, clusterInfo: org.jboss.ha.framework.interfaces.FamilyClusterInfoImpl@1adfc128{familyName=DefaultPartition/jboss.j2ee:jndiName=ejb/ThrottleRuleFacade,service=EJB/H,targets=[org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]],currentViewId=-238527116,isViewMembersInSyncWithViewId=false,cursor=999999999,arbitraryObject=null}, policy=org.jboss.ha.framework.interfaces.RoundRobin@5a67c9
2004-06-24 12:49:44,764 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] throttleRuleHome.create()
2004-06-24 12:49:44,770 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 12:49:44,931 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Init, clusterInfo: org.jboss.ha.framework.interfaces.FamilyClusterInfoImpl@1adfc132{familyName=DefaultPartition/jboss.j2ee:jndiName=ejb/ThrottleRuleFacade,service=EJB/R,targets=[org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]],currentViewId=-238527116,isViewMembersInSyncWithViewId=false,cursor=999999999,arbitraryObject=null}, policy=org.jboss.ha.framework.interfaces.RoundRobin@766a24
2004-06-24 12:49:44,936 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Init, clusterInfo: org.jboss.ha.framework.interfaces.FamilyClusterInfoImpl@1adfc128{familyName=DefaultPartition/jboss.j2ee:jndiName=ejb/ThrottleRuleFacade,service=EJB/H,targets=[org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]],currentViewId=-238527116,isViewMembersInSyncWithViewId=true,cursor=0,arbitraryObject=null}, policy=org.jboss.ha.framework.interfaces.RoundRobin@104c575
2004-06-24 12:49:44,937 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] serviceHome.create()
2004-06-24 12:49:44,938 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 12:49:45,006 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Init, clusterInfo: org.jboss.ha.framework.interfaces.FamilyClusterInfoImpl@5e3bd37b{familyName=DefaultPartition/jboss.j2ee:jndiName=ejb/ServiceFacadeRemote,service=EJB/R,targets=[org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]],currentViewId=-238527116,isViewMembersInSyncWithViewId=false,cursor=999999999,arbitraryObject=null}, policy=org.jboss.ha.framework.interfaces.RoundRobin@97d01f
2004-06-24 12:49:45,009 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Init, clusterInfo: org.jboss.ha.framework.interfaces.FamilyClusterInfoImpl@5e3bd371{familyName=DefaultPartition/jboss.j2ee:jndiName=ejb/ServiceFacadeRemote,service=EJB/H,targets=[org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]],currentViewId=-238527116,isViewMembersInSyncWithViewId=true,cursor=0,arbitraryObject=null}, policy=org.jboss.ha.framework.interfaces.RoundRobin@e0a386
2004-06-24 12:49:45,010 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] Inside block(fromAddress=112112, fromType=2, toAddress=46733117041, toType=1)
2004-06-24 12:49:45,011 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] service.findBySourceAddress
2004-06-24 12:49:45,012 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 12:49:45,407 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] throttleRule.getTrafficType(fromType)
2004-06-24 12:49:45,408 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 12:49:45,460 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] throttleRule.getTrafficType(toType)
2004-06-24 12:49:45,461 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 12:49:45,474 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] throttleRule.findPossibleTrafficTypeConnectionsFromTo(fromType)
2004-06-24 12:49:45,474 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 12:49:45,554 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] throttleRule.createThrottleRule(serviceTO.getId(), tr)
2004-06-24 12:49:45,554 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 12:49:45,650 INFO [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] tr: ThrottleRule with Id: -1 ClassId: se.teligent.smp.throttlerule.model.ThrottleRule SubType: THROTTLERULE Name: SMPP(112112) to Hand-set(46733117041) period 0 units 0 hasChildren: false FromType: TrafficType with type: 2 name: SMPP ToType: TrafficType with type: 1 name: Hand-set Period: 0 Units: 0 FromAddress: 112112 ToAddress: 46733117041 UpdateTS: 1088070585553 ModifyFlag: A TemplateFlag: false ServiceId: 100021 ServiceMediationPLatformId: -1
throttlerule already exists... not doing anything, just sit tight!


//*****jboss0 down, jboss1 up and homeproxies in the cache
2004-06-24 13:10:22,021 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] JNDI lookup java:comp/env/ejb/ServiceFacadeRemote
2004-06-24 13:10:22,162 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] JNDI lookup java:comp/env/ejb/ThrottleRuleFacade
2004-06-24 13:10:22,196 DEBUG [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] throttleRuleHome.create()
2004-06-24 13:10:22,196 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoking on target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
2004-06-24 13:10:22,199 TRACE [org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA] Invoke failed, target=org.jboss.invocation.jrmp.server.JRMPInvoker_Stub[RemoteStub [ref: [endpoint:[172.19.23.30:4545](remote),objID:[ba5c7a:fd5246dd1a:-8000, 3]]]]
java.rmi.ConnectException: Connection refused to host: 172.19.23.30; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:567)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:185)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:171)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:101)
at org.jboss.invocation.jrmp.server.JRMPInvoker_Stub.invoke(Unknown Source)
at org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxyHA.invoke(JRMPInvokerProxyHA.java:189)
at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:96)
at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:45)
at org.jboss.proxy.ejb.HomeInterceptor.invoke(HomeInterceptor.java:173)
at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:85)
at $Proxy2.create(Unknown Source)
at se.teligent.smp.throttlerule.client.ThrottleRuleDelegate.(Unknown Source)
at se.teligent.smp.throttlerule.client.P90ThrottleRuleImpl.block(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:261)
at sun.rmi.transport.Transport$1.run(Transport.java:148)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:144)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:460)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:701)
at java.lang.Thread.run(Thread.java:534)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:305)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:171)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:158)
at java.net.Socket.connect(Socket.java:452)
at java.net.Socket.connect(Socket.java:402)
at java.net.Socket.(Socket.java:309)
at java.net.Socket.(Socket.java:124)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:562)
... 24 more
2004-06-24 13:10:22,205 ERROR [se.teligent.smp.throttlerule.client.ThrottleRuleDelegate] java.rmi.RemoteException: Service unavailable.

[darranl]

The scenario that you are describing is

1 - Start a client
2 - Completely stop the cluster
3 - Start a NEW cluster

You are then expecting the client to be aware of the newly created cluster which is not related to the previous cluster that it was using.

[bill.burke]

A proxy must have at least one available target to communicate to get its list refreshed.

The list gets refreshed on a successful response to a successful EJB invocation.

[bill.burke]

One more thing. Why does it work this way?

We did not want clients to be involved with any multicast protocol as they may be on a WAN or communicate through a firewall