Put a fresh server on the internet with password login enabled, then look at its auth log a little later. You'll find a steady stream of failed logins for root, admin, ubuntu and test, from addresses you've never heard of. Automated bots try this on everything, all day long.

A strong password slows them down. SSH keys make the whole game pointless. They're also faster to use than passwords once they're set up, which is the rare case where the secure option is the convenient one too.

Here's how they work, how to set them up properly, and the small details that separate "I use SSH keys" from "I use SSH keys safely".

How a key pair works

An SSH key is really two keys that belong together:

• The private key stays on your machine and never leaves it. Not in Slack, not in a repo, not in an email to yourself.
• The public key goes on every server you want to log in to. It's safe to share; that's the point of it.

When you connect, the server doesn't ask for a secret. It asks you to prove you hold the private key by signing some data that's unique to this connection. Your machine signs it, the server checks the signature with the public key it has on file, and you're in. The private key itself never crosses the network.

A good analogy is a wax seal. Anyone can be given a picture of your seal to check letters against, but only the person holding the seal itself can press it into new wax.

Step 1: create a key (use ed25519)

ssh-keygen -t ed25519 -C "shiroz-laptop-2025"

A few choices worth making on purpose:

• ed25519 is the modern default: short keys, fast, and strong. Recent OpenSSH versions already use it by default. Use RSA (3072 bits or more) only for old systems that don't support ed25519.
• The comment (-C) should name the device. You'll thank yourself when you're looking at a server's key list trying to work out which one belongs to the laptop you no longer have.
• Set a passphrase. It encrypts the private key on disk, so a stolen laptop or a leaked backup doesn't hand over access to your servers. The agent (below) means you won't type it all day.

Step 2: put the public key on the server

ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]

This appends your public key to ~/.ssh/authorized_keys on the server. That file is simply a list of public keys allowed to log in as that user, one per line.

If you add it by hand, permissions matter. By default, SSH refuses keys if the files are too open:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

"Key not accepted" after a manual setup is very often just this.

Step 3: let the agent hold the key

ssh-agent keeps your decrypted key in memory for the session, so you type the passphrase once instead of on every connection:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

On macOS the keychain can remember it, and on Windows the OpenSSH Authentication Agent service does the same job.

Then save yourself some typing with ~/.ssh/config:

Host prod-web
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Now ssh prod-web is all you need. IdentitiesOnly yes makes SSH offer only that key instead of every key in your agent, which avoids "too many authentication failures" when you have several.

Step 4: turn off passwords

Keys only protect you once password login is switched off. Otherwise the bots can still try passwords. Before you do this, keep your current session open and test a key login in a second terminal. Locking yourself out of a remote server is a rite of passage you can skip.

# /etc/ssh/sshd_config (or a file in /etc/ssh/sshd_config.d/)
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
PubkeyAuthentication yes

Then check the syntax with sudo sshd -t and reload the service (sudo systemctl reload ssh on Ubuntu; it's called sshd on many other distributions).

The gotcha that catches a lot of people: on many distributions, files in sshd_config.d/ are included at the top of the main config, and for sshd the first value wins. Cloud images sometimes ship a file there that sets PasswordAuthentication yes, which quietly overrides your edit further down. Always confirm the setting that's actually in effect:

sudo sshd -T | grep -i passwordauthentication

Habits that keep keys safe

One key per device, not one key everywhere

It's tempting to copy one private key onto your laptop, your desktop and the CI server. Don't. If one device is lost or compromised, you'd have to replace the key everywhere. With one key per device, you delete a single line from authorized_keys and you're done. The descriptive comment from step 1 tells you which line.

Be careful with agent forwarding

ForwardAgent yes lets a server use your local agent, handy for pulling from Git on a server without copying keys there. But while you're connected, anyone with root on that server can use your forwarded agent to log in as you elsewhere. If you just need to hop through a bastion host, use ProxyJump (ssh -J bastion prod-web) instead. It connects through the bastion without exposing your agent to it.

Restrict keys that machines use

Keys for automation (a deploy pipeline, a backup job) can be limited right in authorized_keys, with options like from="..." to allow only certain source addresses, or restrict combined with command="..." to allow only one command. For Git hosting, prefer read-only deploy keys per repository over a personal key with access to everything.

Clean up

Review authorized_keys on your servers from time to time. Keys for people who left, laptops that died and experiments from two years ago tend to collect there. Every unknown line is a door you can't vouch for.

And if you want to go further, OpenSSH supports hardware security keys (ssh-keygen -t ed25519-sk), so the private key lives on a physical device that requires a touch to use.

Checklist

1. Generate an ed25519 key per device, with a passphrase and a clear comment.
2. Install the public key with ssh-copy-id; check permissions if it fails.
3. Use the agent and ~/.ssh/config for convenience.
4. Disable password and root login, then confirm with sshd -T.
5. Avoid agent forwarding; use ProxyJump.
6. Restrict automation keys and prune authorized_keys regularly.

The sshd_config manual is dense but worth a skim; every option above is in there.

Which of these do you already do, and which one are you adding to your servers this week?