when I ssh into RHEL7 AWS-EC2 instance I get 'Permission denied (publickey,gssapi-keex,gssapi-with-mic)'

Question

Ok so I know this question has been asked before. I tried everything that was posted in the closest related question and none of that worked. I was able to log in before but am not able to now. - I have tried ssh-ing in with both root@ipadress and ec2-user@ipaddress neither have worked. When I use root it tells me to use ec2-user. I am ssh-ing from ubuntu 16.04 - I have the correct permissions on my pem file. - Also I do not have a /etc/sshd_special_user file and not sure that adding one will do anything but it does seem to try to authenticate against my local user after failing the first time.
- When I ssh-keygen -f " ~/.ssh/known_hosts" -R xx.xx.xxx.xxx I get mkstemp: No such file or directory. I am getting this even thought the file is there and I am able to open it in vim. I am really confused as to what is going on here and am going to include the extra verbose log file with this question. I read through it and am not sure why it is doing what it is doing.

output from ssh -vvv is

-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1, OpenSSL 1.0.2g 1 Mar 2016

-debug1: Reading configuration data /etc/ssh/ssh_config

-debug1: /etc/ssh/ssh_config line 19: Applying options for *

-debug2: resolving "52.53.159.22" port 22

-debug2: ssh_connect_direct: needpriv 0

-debug1: Connecting to 52.53.159.22 [52.53.159.22] port 22.

-debug1: Connection established.

-debug1: key_load_public: No such file or directory

-debug1: identity file .ssh/RH17-06-11.pem type -1

-debug1: key_load_public: No such file or directory

-debug1: identity file .ssh/RH17-06-11.pem-cert type -1

-debug1: Enabling compatibility mode for protocol 2.0

-debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.1

-debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1

-debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000

-debug2: fd 3 setting O_NONBLOCK

-debug1: Authenticating to 52.53.159.22:22 as 'ec2-user'

-debug3: hostkeys_foreach: reading file "/home/dingofarmers/.ssh/known_hosts"

-debug3: record_hostkey: found key type ECDSA in file /home/dingofarmers/.ssh/known_hosts:21

-debug3: load_hostkeys: loaded 1 keys from 52.53.159.22

-debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521

-debug3: send packet: type 20

-debug1: SSH2_MSG_KEXINIT sent

-debug3: receive packet: type 20

-debug1: SSH2_MSG_KEXINIT received

-debug2: local client KEXINIT proposal

-debug2: KEX algorithms: [email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,ext-info-c

-debug2: host key algorithms: [email protected],[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa

-debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc

-debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected],aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc

-debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1

-debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1

-debug2: compression ctos: none,[email protected],zlib

-debug2: compression stoc: none,[email protected],zlib

-debug2: languages ctos:

-debug2: languages stoc:

-debug2: first_kex_follows 0

-debug2: reserved 0

-debug2: peer server KEXINIT proposal

-debug2: KEX algorithms: [email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1

-debug2: host key algorithms: ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519

-debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,[email protected],[email protected],[email protected],aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected]

-debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,[email protected],[email protected],[email protected],aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,[email protected]

-debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-md5,hmac-sha1,[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96

-debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-md5,hmac-sha1,[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,[email protected],hmac-sha1-96,hmac-md5-96

-debug2: compression ctos: none,[email protected]

-debug2: compression stoc: none,[email protected]

-debug2: languages ctos:

-debug2: languages stoc:

-debug2: first_kex_follows 0

-debug2: reserved 0

-debug1: kex: algorithm: [email protected]

-debug1: kex: host key algorithm: ecdsa-sha2-nistp256

-debug1: kex: server->client cipher: [email protected] MAC: compression: none

-debug1: kex: client->server cipher: [email protected] MAC: compression: none

-debug3: send packet: type 30

-debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

-debug3: receive packet: type 31

-debug1: Server host key: ecdsa-sha2-nistp256 SHA256:qaPtcCft8A+sNZTbFvAsKBPQVvKRqdBYEV93An/SY+w

-debug3: hostkeys_foreach: reading file "/home/dingofarmers/.ssh/known_hosts"

-debug3: record_hostkey: found key type ECDSA in file /home/dingofarmers/.ssh/known_hosts:21

-debug3: load_hostkeys: loaded 1 keys from 52.53.159.22

-debug1: Host '52.53.159.22' is known and matches the ECDSA host key.

-debug1: Found key in /home/dingofarmers/.ssh/known_hosts:21

-debug3: send packet: type 21

-debug2: set_newkeys: mode 1

-debug1: rekey after 134217728 blocks

-debug1: SSH2_MSG_NEWKEYS sent

-debug1: expecting SSH2_MSG_NEWKEYS

-debug3: receive packet: type 21

-debug2: set_newkeys: mode 0

-debug1: rekey after 134217728 blocks

-debug1: SSH2_MSG_NEWKEYS received

-debug2: key: dingofarmers@ubuntu (0x55ce58f7d660), agent

-debug2: key: dingofarmers@ubuntu (0x55ce58f7e920), agent

-debug2: key: .ssh/RH17-06-11.pem ((nil)), explicit

-debug3: send packet: type 5

-debug3: receive packet: type 6

-debug2: service_accept: ssh-userauth

-debug1: SSH2_MSG_SERVICE_ACCEPT received

-debug3: send packet: type 50

-debug3: receive packet: type 51

-debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic

-debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic

-debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password

-debug3: authmethod_lookup gssapi-keyex

-debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password

-debug3: authmethod_is_enabled gssapi-keyex

-debug1: Next authentication method: gssapi-keyex

-debug1: No valid Key exchange context

-debug2: we did not send a packet, disable method

-debug3: authmethod_lookup gssapi-with-mic

-debug3: remaining preferred: publickey,keyboard-interactive,password

-debug3: authmethod_is_enabled gssapi-with-mic

-debug1: Next authentication method: gssapi-with-mic

-debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available

-debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available

-debug1: Unspecified GSS failure. Minor code may provide more information

-debug1: Unspecified GSS failure. Minor code may provide more information No Kerberos credentials available

-debug2: we did not send a packet, disable method

-debug3: authmethod_lookup publickey

-debug3: remaining preferred: keyboard-interactive,password

-debug3: authmethod_is_enabled publickey

-debug1: Next authentication method: publickey

-debug1: Offering RSA public key: dingofarmers@ubuntu

-debug3: send_pubkey_test

-debug3: send packet: type 50

-debug2: we sent a publickey packet, wait for reply

-debug3: receive packet: type 51

-debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic

-debug1: Offering RSA public key: dingofarmers@ubuntu

-debug3: send_pubkey_test

-debug3: send packet: type 50

-debug2: we sent a publickey packet, wait for reply

-debug3: receive packet: type 51

-debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic

-debug1: Trying private key: .ssh/RH17-06-11.pem

-debug3: sign_and_send_pubkey: RSA SHA256:gwWpJTQxOFvICvYC7ILZ8rTnS9F/TjWaYCmxj6toatY

-debug3: send packet: type 50

-debug2: we sent a publickey packet, wait for reply

-debug3: receive packet: type 51

-debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic

-debug2: we did not send a packet, disable method

-debug1: No more authentication methods to try.

Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

Answer 1

Move your key file to .ssh directory

mv path/to/key.pem ~/.ssh/.

Then make sure you gave right permissions to the file

chmod 600 ~/.ssh/key.pem

And connect to server via ssh

ssh -i ~/.ssh/key.pem ec2-user@instance_ip_address

Answer 2

The known_hosts file is typically used for recording the system level ssh keys of the servers you're ssh'ing to, not for private key storage. Is there a particular reason you're putting your private key in there? I would probably completely clear the ~/.ssh directories on client and server (making backups first) and start from scratch to keep things simple.

On the client machine, you typically have the ~/.ssh directory (0700 permissions), ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub (0600 permissions), and the known hosts file that should be auto-populated each time you ssh into a new system. The actual id_rsa filenames aren't critical of course, as long as the ssh client can find them and parse them, but I've never tried putting their contents inside a file that already means something to ssh.

On the server machine, you would also have the .ssh directory (0700 permissions again), and an authorized_keys file (0600 permissions) that contains the text of the id_rsa.pub key (or equivalent) from the client box.

On the client and server:

mkdir -p ~/.ssh
chmod 0700 ~/.ssh

On the client:

ssh-keygen -f ~/.ssh/id_rsa
scp ~/.ssh/id_rsa.pub [email protected]:~/.ssh/authorized_keys

On the server:

chmod 0600 ~/.ssh/authorized_keys

Then try your ssh -vvv again. If that doesn't work run the following command on both client and server and post the results:

sudo grep -i key /etc/ssh/ssh*_config