Public Key Authentication Problems

These notes cover debugging common problems encountered when setting up public key authentication for Secure Shell (SSH) using OpenSSH.

Definition of terms used in this documentation:

client system: the system one types directly on, such as a laptop or desktop system.
server: anything connected to from the client. This includes other servers accessed through the first server connected to.

Never allow root-to-root trust between systems. If required by poorly engineered legacy scripts, limit the from access of the public keys, and if possible only allow specific public keys to run specific commands. Instead, setup named accounts for users or roles, and grant as little root access as possible via sudo.

For more information, see also SSH, The Secure Shell: The Definitive Guide.

Connection Problems

If the connection does not work, try again with the -v -v -v flags specified for verbose output, which may indicate where the problem is. Common sources of problems include:

SSH Protocol 1

A RSA keypair generated will not work if the remote server only supports the old version 1 protocol. The solution in this case is to generate an old version 1 RSA key, and upload the contents of ~/.ssh/identity.pub on the client system into ~/.ssh/authorized_keys on the servers.

client$ ssh-keygen -q -f ~/.ssh/identity -t rsa1

To determine what version of software a SSH server is running, telnet to port 22, then type CONTROL+] to drop to the telnet prompt. The following server offers SSH2 only; other servers would list 1.5 instead of 2.0 for SSH1.

client$ telnet server.example.org 22
…
Escape character is '^]'.
SSH-2.0-OpenSSH_3.8
^]
telnet> quit

Different Server Software

If the server is not running OpenSSH, the documentation for that server will need to be consulted to see whether it supports public key authentication and if so how the public key needs to be stored on the server.

For servers running SSH Secure Shell, the public key will have to be converted, uploaded, and referenced in the ~/.ssh2/authorization file on the server:

client$ ssh-keygen -ef ~/.ssh/id_rsa.pub | \ ssh server.example.org 'cat > .ssh2/public-key1'
client$ ssh server.example.org 'echo Key public-key1 >> .ssh2/authorization'

Permissions

If any of the files (or directories leading up to the files) have permissions set too loose, the connection will fail. Permission errors may be logged on the server side by the sshd(8) daemon.

Authentication refused: bad ownership or modes for directory …

In most cases, potential permission problems can be solved by restricting down access to the SSH configuration files. Permission changes to the home directory might be needed, though restricted rights may break other things, such as a webserver’s access to ~/public_html, for example.

server$ chmod go-w ~/
server$ chmod 700 ~/.ssh
server$ chmod 600 ~/.ssh/authorized_keys

Configuration

Either ssh(1) or sshd(8) or both can be configured to disallow public key authentication. By default, public key authentication is usually enabled. See ssh_config(5) and sshd_config(5) for more information.

Typos

Errors in this category include spelling errors on the configuration file names, or breaking the public key file by wrapping long lines.

If ancient versions of OpenSSH below version 3.0 are still in use, a link will need to be created. Otherwise, the authorized_keys2 should not be used, as support for it may be removed in a future version of OpenSSH.

server$ cd ~/.ssh
server$ cat authorized_keys2 >> authorized_keys
server$ ln -sf authorized_keys authorized_keys2

Agent Test

To test the agent, invoke a shell via the agent. Next, add the key into memory with the ssh-add(1) command and test a connection to the server.

client$ ssh-agent $SHELL
client$ ssh-add -l
The agent has no identities.
client$ ssh-add
Enter passphrase for /…/.ssh/id_rsa: …
Identity added: /…/.ssh/id_rsa (/…/.ssh/id_rsa)
client$ ssh-add -l
1024 48:65:6c:6c:6f:2c:20:77:6f:72:6c:64:21:20:3a:29 /…/.ssh/id_rsa (RSA)
client$ ssh server.example.org
…
server$ exit
client$ exit
client$