2 Replies Latest reply on May 24, 2007 3:03 PM by ojacobson

    JNDI trying to use the wrong host

    ojacobson

      My jboss-4.0.4.GA installation inexplicably stopped working today, and it looks like it's because it started trying to connect to localhost:1099 for JNDI lookups and binding. This is wrong, since jboss is set up to listen only on a specific IP (not 127.0.0.1 or 0.0.0.0), corresponding to foo.dev.example.com.

      Here's the command line used to start jboss:

      /Library/Java/Home/bin/java -server
       -Dprogram.name=startjboss
       -Dsun.rmi.dgc.client.gcInterval=3600000
       -Dsun.rmi.dgc.server.gcInterval=3600000
       -Xms512m -Xmx2048m
       -XX:NewSize=128m -XX:PermSize=96m -XX:MaxPermSize=96m
       -verbose:gc -Xloggc:/sites/foo.dev.example.com/logs/GCOutput_2007.05.23.log
       -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
       -XX:+UseConcMarkSweepGC
       -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=4000
       -Djavax.net.ssl.keyStore=/Library/Java/Home/jre/lib/security/cacerts
       -Djavax.net.ssl.keyStorePassword=******
       -Djavax.xml.soap.MessageFactory=com.sun.xml.messaging.saaj.soap.ver1_1.SOAPMessageFactory1_1Impl
       -Djava.rmi.server.hostname=foo.dev.example.com
       -Djboss.bind.address=foo.dev.example.com
       -Djboss.home.dir=/usr/local/jboss-4.0.4.GA
       -Djboss.server.base.dir=/sites/foo.dev.example.com
       -Djboss.server.home.dir=/sites/foo.dev.example.com
       -Djboss.server.name=
       -Djboss.server.home.url=file:/sites/foo.dev.example.com
       -Djboss.server.lib.url=file:/usr/local/jboss-4.0.4.GA/server/all/lib
       -Djboss.partition.name=localpartition-example
       -Djboss.partition.udpGroup=239.99.150.20
       -Djboss.partition.port=45564
       -classpath /usr/local/jboss-4.0.4.GA/bin/run.jar:/Library/Java/Home/lib/tools.jar:/sites/foo.dev.example.com/conf/something
       org.jboss.Main --host foo.dev.example.com


      During startup, JNDI binds to foo.dev.example.com:

      2007-05-23 18:41:42,130 INFO [org.jboss.naming.NamingService] JNDI bootstrap JNP=foo.dev.example.com/10.6.55.164:1099, RMI=foo.dev.example.com/10.6.55.164:1098, backlog=50, no client SocketFactory, Server SocketFactory=class org.jboss.net.sockets.DefaultSocketFactory
      


      When an MBean goes to look something up in JNDI during startup, I get the following stacktrace:

      2007-05-23 18:42:08,003 DEBUG [org.jnp.interfaces.NamingContext] Failed to connect to localhost:1099
      javax.naming.CommunicationException: Failed to connect to server localhost:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server localhost:1099 [Root exception is
      java.net.ConnectException: Connection refused]]
       at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:269)
       at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1385)
       at org.jnp.interfaces.NamingContext.rebind(NamingContext.java:484)
       at org.jnp.interfaces.NamingContext.rebind(NamingContext.java:477)
       at javax.naming.InitialContext.rebind(InitialContext.java:367)
       at org.jboss.util.naming.NonSerializableFactory.rebind(NonSerializableFactory.java:180)
       at org.jboss.util.naming.NonSerializableFactory.rebind(NonSerializableFactory.java:221)
       at foo.admin.scheduler.mbean.QuartzScheduler.rebind(QuartzScheduler.java:94)
       at foo.admin.scheduler.mbean.QuartzScheduler.startService(QuartzScheduler.java:67)
       at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:289)
       at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:245)
       at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
       at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:978)
       at $Proxy0.start(Unknown Source)
       at org.jboss.system.ServiceController.start(ServiceController.java:417)
       at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
       at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
       at $Proxy4.start(Unknown Source)
       at org.jboss.deployment.SARDeployer.start(SARDeployer.java:302)
       at org.jboss.deployment.MainDeployer.start(MainDeployer.java:1007)
       at org.jboss.deployment.MainDeployer.start(MainDeployer.java:997)
       at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:808)
       at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:771)
       at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
       at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
       at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
       at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
       at $Proxy8.deploy(Unknown Source)
       at org.jboss.deployment.scanner.URLDeploymentScanner.deploy(URLDeploymentScanner.java:421)
       at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentScanner.java:634)
       at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.doScan(AbstractDeploymentScanner.java:263)
       at org.jboss.deployment.scanner.AbstractDeploymentScanner.startService(AbstractDeploymentScanner.java:336)
       at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:289)
       at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:245)
       at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
       at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:978)
       at $Proxy0.start(Unknown Source)
       at org.jboss.system.ServiceController.start(ServiceController.java:417)
       at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
       at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
       at $Proxy4.start(Unknown Source)
       at org.jboss.deployment.SARDeployer.start(SARDeployer.java:302)
       at org.jboss.deployment.MainDeployer.start(MainDeployer.java:1007)
       at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:808)
       at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:771)
       at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:755)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:585)
       at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
       at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
       at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
       at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142)
       at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
       at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
       at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
       at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
       at $Proxy5.deploy(Unknown Source)
       at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:482)
       at org.jboss.system.server.ServerImpl.start(ServerImpl.java:362)
       at org.jboss.Main.boot(Main.java:200)
       at org.jboss.Main$1.run(Main.java:464)
       at java.lang.Thread.run(Thread.java:613)
      Caused by: javax.naming.ServiceUnavailableException: Failed to connect to server localhost:1099 [Root exception is java.net.ConnectException: Connection refused]
       at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:243)
       ... 100 more
      Caused by: java.net.ConnectException: Connection refused
       at java.net.PlainSocketImpl.socketConnect(Native Method)
       at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
       at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:430)
       at java.net.Socket.connect(Socket.java:516)
       at java.net.Socket.connect(Socket.java:466)
       at java.net.Socket.<init>(Socket.java:366)
       at java.net.Socket.<init>(Socket.java:266)
       at org.jnp.interfaces.TimedSocketFactory.createSocket(TimedSocketFactory.java:84)
       at org.jnp.interfaces.TimedSocketFactory.createSocket(TimedSocketFactory.java:77)
       at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:239)
       ... 100 more
      


      Why is it trying to use localhost, given that the same configuration files worked earlier today? (I checked a clean set out of CVS to make sure.)

      And how can I make it stop doing that? Or at least make it work?

        • 1. Re: JNDI trying to use the wrong host
          ojacobson

          I did a little more investigation and compared the JNDI InitialContext environment on two different machines -- one which works and the one which doesn't.

          On the machine that works, new InitialContext()'s environment has no java.naming.provider.url property.

          On the machine that's having this problem, new InitialContext's environment has java.naming.provider.url=localhost. This property is NOT one of the system properties: I dumped System.getProperties().entrySet() immediately before the first failing lookup and java.naming.provider.url is not one of the properties.

          What could be setting this property?

          • 2. Re: JNDI trying to use the wrong host
            ojacobson

            We have met the enemy, and he is us.

            I recently changed the build path in my eclipse environment, causing it to copy test resources into the same place used to package the JAR files used by the server. This caused the test jndi.properties, intended for use from a standalone client, to be included in the server artifacts.