11 Replies Latest reply on Oct 7, 2013 8:59 AM by faridasaraf85

    Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth

    komododave

      We're trying to run the External MQ Client Example on a single child container of a single ensemble node for our Fabric.

       

      The child container has been configured with the native mq profile applied, and is otherwise untouched.

       

      That profile itself is unmodified.

       

      We've also tried the below with a custom broker group as opposed to default; the results are the same; a zookeeper "NoAuth" error is received (see final section of post).

       

      Customisation of External MQ Client Project

       

      We've slightly customised the External MQ Client Example as follows:

       

      1. Two version properties have been updated to the following, since we run Fuse Fabric 7.1 GA (i.e. the 047 RC):

       

      *          <activemq.version>5.7.0.fuse-71-047</activemq.version>

      *          <fabric.version>7.1.0.fuse-047</fabric.version>

       

      2. We provide an explicit username/password at point of creating a Connection:

       

      •               connection = factory.createConnection("admin","admin");

      •               // we've also tried "karaf","karaf" in the line above.

       

      3. We've added the dependency fabric-activemq to the project POM to provide the class FabricDiscoveryAgent. The latest version we could find is 1.0:

       

      *          

      •                       <version>1.0</version>

      *          </dependency>

       

      User configuration in Fabric

       

      We have added these two lines..

       

      •          karaf=karaf,admin

      •          admin=admin,admin

       

      ..to both of these files:

       

      •          $FABRIC/etc/users.properties

      •          $FABRIC/instances/<broker-container-name>/etc/users.properties

       

       

      Running the project -> Zookeeper "NoAuth"

       

      Here's what mvn -e -Pproducer-default prints out when run on the fabric host:

       

      11:34:23 INFO  ******************************

      11:34:23 INFO  Connecting to Fuse MQ Broker using URL: discovery:(fabric:default)

      11:34:23 INFO  ******************************

      11:34:23 INFO  Client environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT

      11:34:23 INFO  Client environment:host.name=ithffu01.ocado.com

      11:34:23 INFO  Client environment:java.version=1.6.0_26

      11:34:23 INFO  Client environment:java.vendor=Sun Microsystems Inc.

      11:34:23 INFO  Client environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.26/jre

      11:34:23 INFO  Client environment:java.class.path=/usr/share/maven2/boot/classworlds.jar

      11:34:23 INFO  Client environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.26/jre/lib/amd64:/usr/lib/jvm/java-6-sun-1.6.0.26/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

      11:34:23 INFO  Client environment:java.io.tmpdir=/tmp

      11:34:23 INFO  Client environment:java.compiler=<NA>

      11:34:23 INFO  Client environment:os.name=Linux

      11:34:23 INFO  Client environment:os.arch=amd64

      11:34:23 INFO  Client environment:os.version=2.6.32-5-amd64

      11:34:23 INFO  Client environment:user.name=root

      11:34:23 INFO  Client environment:user.home=/root

      11:34:23 INFO  Client environment:user.dir=/var/tmp/external-mq-fabric-client-master

      11:34:23 INFO  Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=org.linkedin.zookeeper.client.ZKClient@39fba2af

      11:34:23 DEBUG zookeeper.disableAutoWatchReset is false

      11:34:23 INFO  Opening socket connection to server /0:0:0:0:0:0:0:1:2181

      11:34:23 WARN  SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration.

      11:34:23 INFO  Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration.

      11:34:23 INFO  Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session

      11:34:23 DEBUG Session establishment request sent on localhost/0:0:0:0:0:0:0:1:2181

      11:34:23 INFO  Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x13c6c1b9f0f000b, negotiated timeout = 10000

      11:34:23 DEBUG event: SyncConnected

      11:34:23 DEBUG Reading reply sessionid:0x13c6c1b9f0f000b, packet:: clientPath:null serverPath:null finished:false header:: 1,3  replyHeader:: 1,3028,-101  request:: '/fabric/activemq-clusters/default,F  response::

      11:34:23 DEBUG Reading reply sessionid:0x13c6c1b9f0f000b, packet:: clientPath:null serverPath:null finished:false header:: 2,3  replyHeader:: 2,3028,-101  request:: '/fabric/activemq-clusters/default,F  response::

      11:34:23 DEBUG Reading reply sessionid:0x13c6c1b9f0f000b, packet:: clientPath:null serverPath:null finished:false header:: 3,3  replyHeader:: 3,3028,-101  request:: '/fabric/activemq-clusters,F  response::

      11:34:23 DEBUG Reading reply sessionid:0x13c6c1b9f0f000b, packet:: clientPath:null serverPath:null finished:false header:: 4,3  replyHeader:: 4,3028,0  request:: '/fabric,F  response:: s{2,2,1358934101891,1358934101891,0,3,1,0,0,3,137}

      11:34:23 DEBUG Reading reply sessionid:0x13c6c1b9f0f000b, packet:: clientPath:null serverPath:null finished:false header:: 5,1  replyHeader:: 5,3029,-102  request:: '/fabric/activemq-clusters,,v{s{31,s{'world,'anyone}}},0  response::

      11:34:23 ERROR Could not stop service: org.fusesource.fabric.activemq.FabricDiscoveryAgent@ada6d09. Reason: java.lang.NullPointerException

      java.lang.NullPointerException

              at org.fusesource.fabric.activemq.FabricDiscoveryAgent.stop(FabricDiscoveryAgent.java:180)

              at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:41)

              at org.apache.activemq.transport.discovery.DiscoveryTransport.stop(DiscoveryTransport.java:67)

              at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)

              at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)

              at org.apache.activemq.transport.ResponseCorrelator.stop(ResponseCorrelator.java:132)

              at org.apache.activemq.util.ServiceSupport.dispose(ServiceSupport.java:43)

              at org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:701)

              at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:290)

              at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:191)

              at com.fusesource.examples.activemq.SimpleProducer.main(SimpleProducer.java:61)

              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

              at java.lang.reflect.Method.invoke(Method.java:597)

              at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:297)

              at java.lang.Thread.run(Thread.java:662)

      11:34:23 ERROR javax.jms.JMSException: Could not connect to broker URL: discovery:(fabric:default). Reason: org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode = NoAuth for /fabric/activemq-clusters

       

       

      Any suggestions what might be misconfigured?

        • 1. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
          komododave

          Is anyone able to help with this? We'd like to be able to run applications outside Fabric which contact brokers within Fabric, so it's fairly crucial to work out what's wrong here. We've followed documentation and are using one of your example projects, so wonder whether this is a bug or whether we're not being told of a required configuration change.

           

          We're not sure why the editing of users.properties as described  here wouldn't enable access from the example mq project.

           

          Thank you.

          • 2. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
            komododave

            Here's some output from zookeeper commands, if it's of any use in diagnosing the original "NoAuth" error:

             

            zk:list /fabric/registry/clusters/fusemq/default

            00000000003

             

            zk:get /fabric/registry/clusters/fusemq/default/00000000003

            {"id":"broker1","services":["tcp://${zk:broker1/ip}:56399"],"agent":"broker1"}

             

            zk:get /fabric/authentication/users

            karaf=e7ebf747769e8522b52d1bf47f718788,admin

             

            We've also modified the logging config to output at DEBUG level by changing both

             

                fabric/etc/org.ops4j.pax.logging.cfg

                fabric/instances/broker1/etc/org.ops4j.pax.logging.cfg

             

            Virtually no information is logged of relevance, besides showing us what the process itself outputs regarding session creation.

            • 3. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
              dejanb_dejan

              Hi,

               

              if you're connecting to the secured ZooKeeper, your client need to provide zk password to be used. In the standalone case it's done via

               

              zookeeper.password

               

              system property, like

               

              -Dzookeeper.password=xxx

               

              Can you try to find out zk password that is used and apply it to your test. Also, I just noted that External MQ-Fabric Client Demo uses Fabric 7.0 and I'm not sure it had support for passing zk password. Try also using appropriate 7.1 libs

              • 4. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                komododave

                Thanks for your response, it's great to finally hear a suggestion!

                 

                The modifications specified in the original post are still in place, so the fabric libs version (as specified there) is now 7.1.0.fuse-047 as you recommend.

                 

                We set the zookeeper password upon fabric creation like so:

                 

                •    fabric:create <snip> --zookeeper-password ourpass

                 

                In case it's relevant, the full fabric:create command we use is:

                 

                •    fabric:create --clean --profile fmc --new-user admin --new-user-role admin --new-user-password <admin-pass> --zookeeper-password <zoo-pass> --resolver manualip --manual-ip <host-ip>

                 

                Those parameters listed in  have real values of course.

                 

                I've now modified the customised external mq example to specify under the POM profile producer-default the new password property:

                 

                •    <systemProperty>

                •        <key>zookeeper.password</key>

                •        <value>ourpass</value>

                •    </systemProperty>

                 

                We've re-run and receive exactly the same error. We've tried both admin/admin and karaf/karaf as username/password. As mentioned in the original post, these are defined in both the ensemble node's etc/users.properties and the file of same relative path under the instances/<broker-name> directory.

                 

                Is there any other information we can provide to help figure out what's wrong?

                • 5. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                  dejanb_dejan

                  Hi,

                   

                  I'll try to reproduce it and get back to you soon.

                   

                  Regards,

                  Dejan

                  • 6. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                    dejanb_dejan

                    Hi,

                     

                    I managed to make the example work against secured zookeeper. Take a look at this pull request

                     

                    https://github.com/FuseByExample/external-mq-fabric-client/pull/2/files

                     

                    So basically, you need to configure zookeeper.password, broker.username and broker.password system properties.

                     

                    Can you try it out and let me know if it works for you?

                     

                    Edited by: dejanb on Jan 25, 2013 1:12 PM

                    • 7. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                      pozzalenko_paul.wright

                      Hi Dejan -

                       

                      I have tried this and it works!! Thank you very much. We had to update the SimpleProducer.java as well to pass the broker username/password, but after that it's working fine.

                       

                      We are using  <activemq.version>5.7.0.fuse-71-047</activemq.version> instead of your specified 5.5.1.fuse-7-061, but it still seems to work fine.

                       

                      Follow up question: is there a way we can extract a list of the possible properties we can set via system.properties for Fabric containers, etc ?? I think we could have resolved the issue ourselves reasonably easily if we knew of these properties in the first place, ie: where did you look to figure out you needed to pass in those extra credentials ? There certainly didn't seem to us to be anything in the logs, and I can't recall us setting the admin/admin credentials on the brokers themselves (we just did a 'mq-create' from memory) ?

                       

                      Any insight you can provide would be fab (no pun intended!)

                       

                      Thanks again.

                      • 8. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                        komododave

                        Many thanks indeed for helping us get this to work, dejanb!

                        • 9. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                          dejanb_dejan

                          Hi guys, glad I could help. As for the docs, this is the new change introduced in 7.1 (as ZooKeeper is now secured by default). I'm not sure it documented yet, so feel free to submit Jiras for everything you see missing.

                          • 10. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                            komododave

                            We've just set up our Redhat access portal login and will be sure to send you a support request with future queries.

                             

                            Thanks again!

                            • 11. Re: Fuse MQ in Fabric - External MQ project receives KeeperErrorCode = NoAuth
                              faridasaraf85

                              HI All,

                              I am trying same example and getting following exception while running simple consumer.

                              java.net.SocketException: Address family not supported by protocol family: connect

                                      at sun.nio.ch.Net.connect(Native Method)

                                      at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:532)

                                      at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:203)

                                      at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:213)

                                      at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:946)

                                      at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:973)

                              17:56:13 INFO  Opening socket connection to server 127.0.0.1/127.0.0.1:2181

                               

                               

                              I have added broker user name and pass in pom.xml and added following stuff in java file.

                              String username = System.getProperty("broker.username", "admin");

                                          String password = System.getProperty("broker.password", "admin");

                               

                                          connection = factory.createConnection(username, password);

                               

                              Kindly let me know what else I should change to get rid of above exception.

                              Quick help is appriciated.