SSH, or Secure Shell, is a protocol designed for encrypted communication with remote servers. While traditional password-based methods come with the inherent risk of brute-force attacks or password guessing, SSH key pairs provide a higher level of security. By using SSH, you employ two keys: a public one that you place on your server, and a private one that remains confidential.

What makes this method especially secure? Keys are not only long but also complex, making them resistant to brute-force attempts. Additionally, with key-based access, the private key never actually travels across the network, which reduces the risk of interception. On the server side, only the public key is stored, meaning even if an attacker accesses it, they can't reverse-engineer the private key. Furthermore, even if a private key falls into the wrong hands, it can be safeguarded with a passphrase, adding an extra layer of protection.

In this guide, we will walk through the steps of setting up SSH keys on Ubuntu to facilitate a more safe and secure way for us to interact with remote servers.

Creating the key pair

The ssh-keygen command is a tool provided by OpenSSH to generate, manage, and convert authentication keys for SSH (Secure SHell). We can use this command is to create a new pair of private and public keys for SSH authentication.

By default, the private key is generated and saved to the file ~/.ssh/id_rsa in your home directory. It's essential to keep this private key secure, as unauthorized access to it can compromise any system where its corresponding public key is used for authentication. Alongside the private key, a public key is also generated and saved with a .pub extension. We can share this public key with any system where you want to use key-based authentication.

Let’s run the command in our local terminal:

Or, for a larger (more secure) 4096-bit key:

We can also append a comment to the public key using the -C flag as follows:

Comments can be used to describe the purpose of the key, the device it belongs to, or any other identifying information. They're also visible to anyone who can see the public key, so avoid including sensitive information in the comment.

The above commands will output the following:

After starting the command, it will prompt us for a location to save these keys, and we can simply press ENTER to accept the default locations. Additionally, we’ll have an option to add a passphrase to your private key for added security, which is recommend and helps in encrypting the private key which is stored locally.

You now have a public and private key that you can use to authenticate.

Add the SSH key to the SSH agent cache

The SSH agent is a program that runs in the background and acts as a "key manager" for SSH. When we load a private key into the SSH agent, the agent handles the process of private key-based authentication on our behalf without requiring us to re-enter the key's passphrase each time we use the key.

Check if the key has already added with the following command:

If for any reason our key is not listed, we can manually add it to the cache:

Copying the public key to your Ubuntu server

If we want to securely connect to our server using SSH keys, the first step is to put our public key on that server. Most times, when we're setting up a new server or cloud computer, there will be a step where we’re asked to add our SSH public key.

Let’s view the contents of our SSH public key on our local machine — replaceid_rsa.pub with your own filename if you used something other than the default. Note: the .pub file extension indicates that this is indeed our public key (we never want to share our private key).

There are a number of methods for adding the public key to a server, and your service provider should have documentation for how to do-so. In my particular case I am using Ploi to provision my Digital Ocean VPS, so I will simply copy the output from the above command and paste it via Ploi’s dashboard.

Authenticating to Your Ubuntu Server Using SSH Keys

If we have successfully added your public key to your server, we should be able to log into the remote host without providing the remote account’s password.

When connecting for the first time, we will be prompted with something like the following:

Before we type "yes", we need to verify that the fingerprint provided in the message matches the fingerprint of the server we’re trying to connect to. If you have physical access to the server or can access it through another trusted method, you can display its SSH fingerprint using:

Replace ssh_host_ed25519_key.pub with the appropriate public key file if a different type of key is in use (like RSA or ECDSA).

If you're using a cloud provider or a hosting service, they might provide the SSH fingerprint of your server in their control panel or documentation. Compare the fingerprint shown there with the one you get when trying to connect.

Once verified, enter “yes” and press ENTER. This will save the server's public key to your ~/.ssh/known_hosts file and complete the connection.

Utilizing the config File for Simplified SSH Access

The SSH config file, located in our ~/.ssh/ directory, offers a powerful way to streamline and customize how we initiate SSH connections. Instead of typing out the full SSH command with all its options each time, we can set up predefined shortcuts and configurations for each server in this file.

The configuration provided below is a typical entry for a server (Port 22 is the default and may be omitted):

With the above configuration in place, we can easily connect to our server by typing:

Disabling Password Authentication on Your Server

As a final step, it is typically considered “best practice” to disable password authentication for the server once SSH key-based access has been set up. This reduces the risk of brute-force attacks. Refer to your particular provider’s documentation regarding this topic, or if you're using Digital Ocean like me, you can follow their guide to disable password authentication.