-
1. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
stevef1uk99 Jun 5, 2014 4:58 PM (in response to akhettar)I have tried and failed to set up a connection with ssh and this version. As a newbie I assumed I had done something wrong. I noticed that the rsa example key in the key.properties files appears to have a bit at the front and id at the back removed, which I did.
-
2. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Jun 6, 2014 4:50 AM (in response to stevef1uk99)Thanks for the reply. Did you say that you have solved the issue by tampering the key.properties?
-
3. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
ffang Jun 8, 2014 8:32 PM (in response to akhettar)Hi,
Could you please try to check the .ssh/known_hosts and remove the line for managed container host to see if it helps?
Freeman
-
4. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Jun 19, 2014 7:07 AM (in response to ffang)HI Freeman
I have removed all the entries from known_hosts but still getting the same issue again and again. We are using fabric-1.1.0.CR1 version by the way. We provision bundles and then we lose the ability to either connect to the root or the managed container. The error below was as result of trying to connect to root container ( managed container is not used, all features are installed in the root container). The services are still being accessed via their corresponding bundles so the server is up and running but can't ssh to it. Note the server is running locally, not remotely yet.
To be frank, this is causing us lots of concerns here as to whether Fabric is fit for the production environment. Is this a known issue?
khettar:bin akhettar$ ./client -u admin -p admin
367 [pool-2-thread-2] WARN org.apache.sshd.client.keyverifier.AcceptAllServerKeyVerifier - Server at /0.0.0.0:8101 presented unverified key:
Authentication failure
akhettar:bin akhettar$
-
5. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Jul 7, 2014 11:12 AM (in response to akhettar)Hi
This is still an issue for us. I have come across this bug(https://issues.jboss.org/browse/FABRIC-357) which seems to be relevant to the issue I am describing here. It has been resolved in 7.2.0.redhat-60 version and we are using Fabri8 1.1.0.CR1 version. Any direction on this would be really appreciated.
Ayache
-
6. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
sonicaaaa Jul 29, 2014 9:54 AM (in response to akhettar)Does it fail evewn if you use ssh to connect?
ex.
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=password -p 8101 -l admin localhost
+ can you add logs and steps to reproduce the problem?
-
7. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Jul 29, 2014 11:34 AM (in response to sonicaaaa)The user is still present in etc/users.properties. I have inserted a screenshot see above showing that If I deploy bundles into managed container then i lose the ability to connect via ssh or Hawtio. If I click on the threads tab I can see one thread blocked and this is only shown when I hit the ssh issue.
Hi Paolo
Yes it does fail with the above ssh command - see below. I can't see anything obvious in the karaf log when I get Permission denied error. This problem is intermittent, I have seen it happening quite often when a connection to a third party resource is broken such as ActiveMQ, or Hbase etc. I have seen some posts on the net people complaining about the SSH issue and I am really wondering if you guys have come across it.
Ayache
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=password -p 8101 -l admin localhost
Warning: Permanently added '[localhost]:8101' (DSA) to the list of known hosts.
Authenticated with partial success.
admin@localhost's password:
Authenticated with partial success.
Permission denied, please try again.
admin@localhost's password:
Authenticated with partial success.
Permission denied, please try again.
admin@localhost's password:
-
8. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
sonicaaaa Jul 29, 2014 11:20 AM (in response to akhettar)Ok. I will take the opportunity to use you as a human debugger. Can you try to use the same command adding a {{-vv}} to have extended debugging informations of ssh side.
And at the same time could you check if your user is still at his place and enabled in
{{fabric8-karaf-1.1.0-SNAPSHOT/etc/users.properties}}
-
9. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Jul 29, 2014 12:17 PM (in response to sonicaaaa)with -v option:
akhettar:etc akhettar$ ssh -v -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=password -p 8101 -l admin localhost
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/akhettar/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug1: Connecting to localhost [::1] port 8101.
debug1: Connection established.
debug1: identity file /Users/akhettar/.ssh/id_rsa type 1
debug1: identity file /Users/akhettar/.ssh/id_rsa-cert type -1
debug1: identity file /Users/akhettar/.ssh/id_dsa type -1
debug1: identity file /Users/akhettar/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version SSHD-CORE-0.9.0
debug1: no match: SSHD-CORE-0.9.0
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug1: kex: client->server aes128-ctr hmac-sha1 none
debug1: sending SSH2_MSG_KEXDH_INIT
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug1: Server host key: DSA d4:f3:ee:bd:ce:b4:b7:6f:91:03:2d:79:fc:09:6f:52
debug1: checking without port identifier
Warning: Permanently added '[localhost]:8101' (DSA) to the list of known hosts.
debug1: ssh_dss_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
debug1: Next authentication method: password
admin@localhost's password:
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
Permission denied, please try again.
-
10. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
sonicaaaa Jul 29, 2014 1:31 PM (in response to akhettar)Still no clue.
This jira describes a bug that looks similar to what reported:
https://issues.apache.org/jira/browse/SSHD-254
Fabirc8 relies on v0.9.0
let's try to see if we can totally suppress pubkey auth and swith to keyboard-interactive:
ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=keyboard-interactive -o PubkeyAuthentication=no -i /dev/null -p 8101 -l admin -vvv
-
11. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Jul 30, 2014 9:16 AM (in response to sonicaaaa)Hi Paolo
Here is the output with debug on:
akhettar:bin akhettar$ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=password -p 8101 -l admin localhost -vvv
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/akhettar/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to localhost [::1] port 8101.
debug1: Connection established.
debug3: Incorrect RSA1 identifier
debug3: Could not load "/Users/akhettar/.ssh/id_rsa" as a RSA1 public key
debug1: identity file /Users/akhettar/.ssh/id_rsa type 1
debug1: identity file /Users/akhettar/.ssh/id_rsa-cert type -1
debug1: identity file /Users/akhettar/.ssh/id_dsa type -1
debug1: identity file /Users/akhettar/.ssh/id_dsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version SSHD-CORE-0.9.0
debug1: no match: SSHD-CORE-0.9.0
debug2: fd 5 setting O_NONBLOCK
debug3: put_host_port: [localhost]:8101
debug3: load_hostkeys: loading entries for host "[localhost]:8101" from file "/dev/null"
debug3: load_hostkeys: loaded 0 keys
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-dss
debug2: kex_parse_kexinit: aes128-ctr
debug2: kex_parse_kexinit: aes128-ctr
debug2: kex_parse_kexinit: hmac-sha1
debug2: kex_parse_kexinit: hmac-sha1
debug2: kex_parse_kexinit: none
debug2: kex_parse_kexinit: none
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-sha1
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug2: mac_setup: found hmac-sha1
debug1: kex: client->server aes128-ctr hmac-sha1 none
debug2: dh_gen_key: priv key bits set: 155/320
debug2: bits set: 1053/2048
debug1: sending SSH2_MSG_KEXDH_INIT
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug1: Server host key: DSA e2:70:cd:93:71:4b:ca:f7:01:12:cf:1d:68:6f:fb:b6
debug3: put_host_port: [::1]:8101
debug3: put_host_port: [localhost]:8101
debug3: load_hostkeys: loading entries for host "[localhost]:8101" from file "/dev/null"
debug3: load_hostkeys: loaded 0 keys
debug1: checking without port identifier
debug3: load_hostkeys: loading entries for host "localhost" from file "/dev/null"
debug3: load_hostkeys: loaded 0 keys
Warning: Permanently added '[localhost]:8101' (DSA) to the list of known hosts.
debug2: bits set: 997/2048
debug1: ssh_dss_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /Users/akhettar/.ssh/id_rsa (0x7fac9140b9b0),
debug2: key: /Users/akhettar/.ssh/id_dsa (0x0),
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
debug3: start over, passed a different list keyboard-interactive,password,publickey
debug3: preferred password
debug3: authmethod_lookup password
debug3: remaining preferred:
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
admin@localhost's password:
debug3: packet_send2: adding 64 (len 57 padlen 7 extra_pad 64)
debug2: we sent a password packet, wait for reply
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
Permission denied, please try again.
admin@localhost's password:
debug3: packet_send2: adding 64 (len 55 padlen 9 extra_pad 64)
debug2: we sent a password packet, wait for reply
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
Permission denied, please try again.
admin@localhost's password:
debug3: packet_send2: adding 64 (len 55 padlen 9 extra_pad 64)
debug2: we sent a password packet, wait for reply
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (keyboard-interactive,password,publickey).
-
12. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
sonicaaaa Jul 30, 2014 11:21 AM (in response to sonicaaaa)Hi can you try the last command I pasted?
-
13. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Jul 30, 2014 11:39 AM (in response to sonicaaaa)Hi
I had to add the hostname at the end for it to work:
akhettar:scripts akhettar$ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o PreferredAuthentications=keyboard-interactive -o PubkeyAuthentication=no -i /dev/null -p 8101 -l admin -vvv localhost
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/akhettar/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: /etc/ssh_config line 102: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to localhost [::1] port 8101.
debug1: Connection established.
debug3: Truncated RSA1 identifier
debug3: Could not load "/dev/null" as a RSA1 public key
debug1: identity file /dev/null type -1
debug1: identity file /dev/null-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version SSHD-CORE-0.9.0
debug1: no match: SSHD-CORE-0.9.0
debug2: fd 5 setting O_NONBLOCK
debug3: put_host_port: [localhost]:8101
debug3: load_hostkeys: loading entries for host "[localhost]:8101" from file "/dev/null"
debug3: load_hostkeys: loaded 0 keys
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-dss
debug2: kex_parse_kexinit: aes128-ctr
debug2: kex_parse_kexinit: aes128-ctr
debug2: kex_parse_kexinit: hmac-sha1
debug2: kex_parse_kexinit: hmac-sha1
debug2: kex_parse_kexinit: none
debug2: kex_parse_kexinit: none
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-sha1
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug2: mac_setup: found hmac-sha1
debug1: kex: client->server aes128-ctr hmac-sha1 none
debug2: dh_gen_key: priv key bits set: 145/320
debug2: bits set: 1013/2048
debug1: sending SSH2_MSG_KEXDH_INIT
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug1: Server host key: DSA e2:70:cd:93:71:4b:ca:f7:01:12:cf:1d:68:6f:fb:b6
debug3: put_host_port: [::1]:8101
debug3: put_host_port: [localhost]:8101
debug3: load_hostkeys: loading entries for host "[localhost]:8101" from file "/dev/null"
debug3: load_hostkeys: loaded 0 keys
debug1: checking without port identifier
debug3: load_hostkeys: loading entries for host "localhost" from file "/dev/null"
debug3: load_hostkeys: loaded 0 keys
Warning: Permanently added '[localhost]:8101' (DSA) to the list of known hosts.
debug2: bits set: 1050/2048
debug1: ssh_dss_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /Users/akhettar/.ssh/id_rsa (0x7fdcd1f00020),
debug2: key: /dev/null (0x0), explicit
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
debug3: start over, passed a different list keyboard-interactive,password,publickey
debug3: preferred keyboard-interactive
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred:
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req
Password authentication
debug2: input_userauth_info_req: num_prompts 1
debug3: packet_send2: adding 32 (len 19 padlen 13 extra_pad 64)
Authenticated with partial success.
debug1: Authentications that can continue: keyboard-interactive,password,publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (keyboard-interactive,password,publickey).
-
14. Re: Can't connect to managed container on fabric8 1.1.0-SNAPSHOT.
akhettar Aug 1, 2014 10:47 AM (in response to akhettar)Hi
I was playing with the startup orders set in etc/startup.properties, by setting org/apache/sshd/sshd-core/0.9.0/sshd-core-0.9.0.jar=92 instead of '30' as default. The ssh issue is less of problem but still occurring from time to time.
Error for managed containers
admin@root>container-connect main-container
Connected
Password authentication failed
SSH Login for main-container: admin
SSH Password for admin@main-container:
Connected
Password authentication failed
Error executing command: Failed to authenticate.
admin@root>