1 Reply Latest reply on Apr 15, 2005 7:19 PM by mmindenhall

    ConnectionManager losing track of connections?

    mmindenhall

      Hi,

      I'm working on an unusual resource adapter. Rather than a single, discrete "EIS" system, I need to connect from an app server to tens, perhaps hundreds of thousand of clients (which are set top boxes in a cable television environment). These connections are sporadic and short-lived, so I do not retain open connections in the connection pool. Doing so would be extremely wasteful -- we would have a connection pool of n connections (where n is the number of active clients on the cable network -- 10s or 100s of thousands), each with an open socket to a remote STB.

      Instead, I have chosen to implement connections as little more than a wrapper around a SocketChannel, which gets de-referenced and then re-created each time the connection is used (this is because SocketChannels cannot be reopened once they are closed). Thus, all connections in the connection pool are equivalent, and can be used to connect to any client on the network.

      Here is the basic flow.

      1. Create a new ConnectionRequestInfo with info required to establish the connection (IP/port, etc.), and pass it into ConnectionFactory.getConnection.

      2. The connection returned (whether new or from the pool) uses the ConnectionRequestInfo parameters to create and open a new SocketChannel to the remote STB.

      3. Use the connection to exchange messages with the remote STB.

      4. When finished, close the connection, which in turn closes and de-references the SocketChannel, and ultimately returns the connection to the pool.

      All of that may or may not be relevant to the issue at hand, but at least gives some context for what I am trying to accomplish. Everything I describe above is implemented and working for a single connection.

      So here is the problem: I can obtain a connection, but when I am finished with it and close it, I get an error indicating (I think) that I'm returning a connection that was never known by the ConnectionManager. Trying to figure out why this was happening, I stumbled across an exchange between Adrian and Scott Stark, which seemed to indicate that the ConnectionManager was keying connections based on the Subject. Here is a link to the full exchange followed by a direct quote:
      http://www.jboss.org/index.html?module=bb&op=viewtopic&t=60104


      "scott.stark@jboss.org" wrote:
      That definitely breaks the connection pooling as the selection of the InternalManagedConnectionPool is based on a Subject that does not have the PasswordCredential, but the pools map key is a Subject with this added in. There would have to be another pooling criteria key that only validated the equality of the Subject principals for this type of change to be effective, but then such a pool would not allow for the same user obtaining different connections based on different db login credentials.


      In my implementation (component-managed rather than container-managed security), I am being passed a null Subject. So if the connection pool is in fact keying connections with Subject, could a null Subject cause it to lose track of the connection? If this is the case, it would seem to break component-managed sign-in, as described in "Option C" in sections 9.1.8.1 and 9.1.9 of the JCA 1.5 spec. In both of these options, it is correct for the container to pass a null Subject!

      Any help or suggestions would be greatly appreciated! The log output I get from creating and closing a connection is below.


      Calling getConnection on my connection factory:

      2005-04-14 18:09:22,072 INFO [com.xxx.cma.server.message.ra.outbound.CMAConnectionFactoryImpl] getConnection called with ConnectionSpec.
      2005-04-14 18:09:22,072 TRACE [org.jboss.resource.connectionmanager.NoTxConnectionManager] subject: null
      2005-04-14 18:09:22,072 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] internalRegisterPool: registering pool with interval 900000 old interval: 9223372036854775807
      2005-04-14 18:09:22,072 DEBUG [org.jboss.resource.connectionmanager.IdleRemover] internalRegisterPool: about to notify thread: old next: 1113524212072, new next: 1113524212072
      2005-04-14 18:09:22,072 INFO [com.xxx.cma.server.message.ra.outbound.ManagedConnectionFactoryImpl] createManagedConnection(Subject, ConnectionRequestInfo) called.
      2005-04-14 18:09:22,072 INFO [com.xxx.cma.server.message.ra.outbound.ManagedConnectionFactoryImpl] Subject contains: null
      2005-04-14 18:09:22,088 DEBUG [com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl] Constructor called (MCF, subject, cxReqInfo).
      2005-04-14 18:09:22,088 DEBUG [com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl] Adding connection event listener on id0
      2005-04-14 18:09:22,088 TRACE [org.jboss.resource.connectionmanager.JBossManagedConnectionPool] supplying new ManagedConnection: org.jboss.resource.connectionmanager.NoTxConnectionManager$NoTxConnectionEventListener@129efd0[state=NORMAL mc=com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl@3f6843 handles=0 lastUse=1113523762088 permit=false trackByTx=false mcp=org.jboss.resource.connectionmanager.JBossManagedConnectionPool$OnePool@d6466f context=org.jboss.resource.connectionmanager.InternalManagedConnectionPool@1aa3554]
      2005-04-14 18:09:22,088 TRACE [org.jboss.resource.connectionmanager.JBossManagedConnectionPool] Getting connection from pool org.jboss.resource.connectionmanager.NoTxConnectionManager$NoTxConnectionEventListener@129efd0[state=NORMAL mc=com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl@3f6843 handles=0 lastUse=1113523762088 permit=true trackByTx=false mcp=org.jboss.resource.connectionmanager.JBossManagedConnectionPool$OnePool@d6466f context=org.jboss.resource.connectionmanager.InternalManagedConnectionPool@1aa3554] [InUse/Available/Max]: [1/19/20]
      2005-04-14 18:09:22,088 INFO [com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl] getConnection called on id 0
      2005-04-14 18:09:22,103 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] Loading properties from XML.
      2005-04-14 18:09:22,103 INFO [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] New CMAConnectionImpl created: 0
      2005-04-14 18:09:22,119 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] InetSocketAddress info:
      Hostname: MINDEN-GX280
      Address: MINDEN-GX280/192.168.168.50
      isUnresolved: false
      Port: 8990
      2005-04-14 18:09:22,119 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] Opening SocketChannel.
      2005-04-14 18:09:22,119 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] Calling setReuseAddress(true)
      2005-04-14 18:09:22,119 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] Calling setSoLinger(false,0)
      2005-04-14 18:09:22,119 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] Calling m_channel.connect to: MINDEN-GX280/192.168.168.50:8990
      2005-04-14 18:09:22,119 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] SocketChannel details:
      Connected: true
      
      2005-04-14 18:09:22,119 DEBUG [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] calling configureBlocking(false)
      2005-04-14 18:09:22,119 INFO [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] CMAConnection ready to use (after open(...) ): 0
      2005-04-14 18:09:22,119 TRACE [org.jboss.resource.connectionmanager.CachedConnectionManager] registering connection from org.jboss.resource.connectionmanager.NoTxConnectionManager@119087d, connection : com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl@bfc8d0, key: org.jboss.resource.connectionmanager.CachedConnectionManager$KeyConnectionAssociation@88ee03
      2005-04-14 18:09:22,135 INFO [com.xxx.cma.server.message.ra.ejb.DynamicRouterBean] Sending message to: localhost/127.0.0.1:8990
      2005-04-14 18:09:22,135 INFO [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] Entering sendMessage method on id 0
      



      Closing the connection, which should result in it landing back in the connection pool:

      2005-04-14 18:09:22,150 INFO [com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl] Closing CMAConnectionImpl id: 0
      2005-04-14 18:09:22,150 INFO [com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl] sending connection event (id 0): javax.resource.spi.ConnectionEvent[source=com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl@3f6843]
      2005-04-14 18:09:22,150 TRACE [org.jboss.resource.connectionmanager.CachedConnectionManager] unregistering connection from org.jboss.resource.connectionmanager.NoTxConnectionManager@119087d, object: null, key: org.jboss.resource.connectionmanager.CachedConnectionManager$KeyConnectionAssociation@88ee03
      2005-04-14 18:09:22,150 INFO [org.jboss.resource.connectionmanager.NoTxConnectionManager] Throwable from unregisterConnection
      java.lang.IllegalStateException: Trying to return an unknown connection2! null
       at org.jboss.resource.connectionmanager.CachedConnectionManager.unregisterConnection(CachedConnectionManager.java:374)
       at org.jboss.resource.connectionmanager.NoTxConnectionManager$NoTxConnectionEventListener.connectionClosed(NoTxConnectionManager.java:93)
       at com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl.sendConnectionEvent(ManagedConnectionImpl.java:189)
       at com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl.close(CMAConnectionImpl.java:254)
       at com.xxx.cma.server.message.ra.ejb.DynamicRouterBean.onMessage(DynamicRouterBean.java:182)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.invocation.Invocation.performCall(Invocation.java:345)
       at org.jboss.ejb.MessageDrivenContainer$ContainerInterceptor.invoke(MessageDrivenContainer.java:475)
       at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:185)
       at org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor.invoke(MessageDrivenInstanceInterceptor.java:87)
       at org.jboss.ejb.plugins.CallValidationInterceptor.invoke(CallValidationInterceptor.java:48)
       at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:105)
       at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:313)
       at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:146)
       at org.jboss.ejb.plugins.RunAsSecurityInterceptor.invoke(RunAsSecurityInterceptor.java:94)
       at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:192)
       at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
       at org.jboss.ejb.MessageDrivenContainer.internalInvoke(MessageDrivenContainer.java:389)
       at org.jboss.ejb.Container.invoke(Container.java:870)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:144)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:249)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:642)
       at org.jboss.invocation.local.LocalInvoker$MBeanServerAction.invoke(LocalInvoker.java:155)
       at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:104)
       at org.jboss.invocation.InvokerInterceptor.invokeLocal(InvokerInterceptor.java:115)
       at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:101)
       at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
       at org.jboss.ejb.plugins.inflow.MessageEndpointInterceptor.delivery(MessageEndpointInterceptor.java:237)
       at org.jboss.ejb.plugins.inflow.MessageEndpointInterceptor.invoke(MessageEndpointInterceptor.java:117)
       at org.jboss.proxy.ClientMethodInterceptor.invoke(ClientMethodInterceptor.java:59)
       at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:91)
       at $Proxy61.onMessage(Unknown Source)
       at com.xxx.cma.server.message.ra.inbound.Dispatcher.run(Dispatcher.java:116)
       at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:200)
       at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:261)
       at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:743)
       at java.lang.Thread.run(Thread.java:595)
      2005-04-14 18:09:22,150 INFO [org.jboss.resource.connectionmanager.NoTxConnectionManager] Unregistered handle that was not registered! null for managedConnection: com.xxx.cma.server.message.ra.outbound.ManagedConnectionImpl@3f6843
      2005-04-14 18:09:22,150 TRACE [org.jboss.resource.connectionmanager.NoTxConnectionManager] unregisterConnection: 1 handles left
      2005-04-14 18:09:22,150 TRACE [org.jboss.resource.connectionmanager.CachedConnectionManager] popped object: org.jboss.resource.connectionmanager.CachedConnectionManager$KeyConnectionAssociation@88ee03
      2005-04-14 18:09:22,150 INFO [org.jboss.resource.connectionmanager.CachedConnectionManager] Closing a connection for you. Please close them yourself: com.xxx.cma.server.message.ra.outbound.CMAConnectionImpl@bfc8d0
      java.lang.Exception: STACKTRACE
       at org.jboss.resource.connectionmanager.CachedConnectionManager.registerConnection(CachedConnectionManager.java:320)
       at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:477)
       at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:838)
       at com.xxx.cma.server.message.ra.outbound.CMAConnectionFactoryImpl.getConnection(CMAConnectionFactoryImpl.java:90)
       at com.xxx.cma.server.message.ra.ejb.DynamicRouterBean.onMessage(DynamicRouterBean.java:155)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.invocation.Invocation.performCall(Invocation.java:345)
       at org.jboss.ejb.MessageDrivenContainer$ContainerInterceptor.invoke(MessageDrivenContainer.java:475)
       at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:185)
       at org.jboss.ejb.plugins.MessageDrivenInstanceInterceptor.invoke(MessageDrivenInstanceInterceptor.java:87)
       at org.jboss.ejb.plugins.CallValidationInterceptor.invoke(CallValidationInterceptor.java:48)
       at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:105)
       at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:313)
       at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:146)
       at org.jboss.ejb.plugins.RunAsSecurityInterceptor.invoke(RunAsSecurityInterceptor.java:94)
       at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:192)
       at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
       at org.jboss.ejb.MessageDrivenContainer.internalInvoke(MessageDrivenContainer.java:389)
       at org.jboss.ejb.Container.invoke(Container.java:870)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:144)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:249)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:642)
       at org.jboss.invocation.local.LocalInvoker$MBeanServerAction.invoke(LocalInvoker.java:155)
       at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:104)
       at org.jboss.invocation.InvokerInterceptor.invokeLocal(InvokerInterceptor.java:115)
       at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:101)
       at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
       at org.jboss.ejb.plugins.inflow.MessageEndpointInterceptor.delivery(MessageEndpointInterceptor.java:237)
       at org.jboss.ejb.plugins.inflow.MessageEndpointInterceptor.invoke(MessageEndpointInterceptor.java:117)
       at org.jboss.proxy.ClientMethodInterceptor.invoke(ClientMethodInterceptor.java:59)
       at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:91)
       at $Proxy61.onMessage(Unknown Source)
       at com.xxx.cma.server.message.ra.inbound.Dispatcher.run(Dispatcher.java:116)
       at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:200)
       at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:261)
       at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:743)
       at java.lang.Thread.run(Thread.java:595)
      



        • 1. Ignore the above!
          mmindenhall

          Nevermind! I figured it out. I was not calling the ConnectionEvent.setConnectionHandle method to associate the Connection with the event when closing the connection. Just a case of being confused and not reading the JCA spec and javadoc carefully. Apparently, JBoss calls the ConnectionEvent.getConnectionHandle to determine which connection is being closed and can be returned to the connection pool.

          Since the connection handle is REQUIRED to be set via setConnectionHandle when closing a connection and notifying ConnectionEventListeners, it would make sense for JBoss to enforce this by logging a meaningful warning or error if this is not done. It should be easy to check for null when retrieving the handle via ConnectionEvent.getConnectionHandle within the ConnectionEventListener.connectionClosed method, and log an appropriate message to spare anyone else from the same oversight in the future!