Yep, just did.. A lot of those devices don't even exist anymore but the keys exist lol.

You should encrypt your ssh keys anyway, and you should encrypt anything sensitive you are backing up to a cloud.

Private keys should never leave the device where they are created.

So no backups?

Correct. Private keys should never be backed up. Instead, should you need a backup, you should create a distinct key for that purpose.

That's a great plan until you're locked out of all your devices with no backup.

I think the implication is that you should own multiple client devices capable of SSHing into things, each with their own SSH keypair; and every SSH host you interact with should have multiple of your devices’ keypairs registered to it.

Right, and to never backup the keys which means losing of all your devices means you can't possibly recover.

Tuna-Fish said that instead of backing up the keys from your devices, you should create a specific backup key that is only ever used in case you lose access to all your devices.

This is indeed best practice because it allows you to alert based on key: if you receive a login on a machine with your backup key, but you haven't lost your devices, then you know your backup was compromised. If you take backups of your regular key then it would be much more difficult to notice a problem.

My point was that one of the devices would be your (cold) backup — you'd e.g. get an (ideally passphrase-protectable) smart-card; read off its pubkey; register that pubkey with all your remote systems/services; and then put the smart-card itself into a fire safe / safe-deposit box at a bank / leave it in trust with your lawyer / etc.

Note that you would never need to go get the smart-card just to perform incremental registration between it and a new remote host/service. You just need its pubkey, which can live in your password manager or wherever.

And yet, if your house burns down, you can go get that smart-card, and use it to get back into all your services.

And yet also, unlike a backup of another of your keys, if you find out that someone broke into your house and stole your safe, or robbed your bank, etc, then you can separately revoke the access of the pubkey associated with the smart-card, without affecting / requiring the rolling of the keys associated with your other devices. (And the ideal additional layer of passphrase protection for the card, gives you a time window to realize your card has been taken, and perform this revocation step, before the card can be cracked and used.)

Indeed, as the sibling comment mentions, this is vaguely similar to a (symmetrically passphrase-encrypted) backup of a unique extra KPI keypair onto a USB stick or somesuch.

The major difference, though, is that because a backup of a key is truly "just data", an attacker can copy off the encrypted file (or image the raw bytes of the encrypted USB disk), and then spawn 10000 compute instances to attempt to crack that encrypted file / disk image.

Whereas, even when in possession of the smart-card, the attacker can't make 10000 copies of the data held in the smart-card. All they can do is attack the single smart-card they have — where doing so may in turn cause the smart-card to delete said data, or to apply exponential-backoff to failed attempts to activate/use the key material. The workflow becomes less like traditional password cracking, and more like interrogating a human (who has been explicitly trained in Resistance-to-Interrogation techniques.)

To me that just sounds like creating obstacles for myself to get access to my system when I desperately need to. I keep a backup of my work pc keys on Google Drive and I have zero anxiety about that.

You can have backup private keys, they don't have to be copies of some other private keys.

Actually, you shouldn’t. You probably use an easy-to-remember password on SSH keys since you have to type them often, but that also means you’re storing one of your (let’s face it, the primary) password you have in a single file, readable to every executable your run under your account. And that means you’re one exfil away from not only getting your SSH keys compromised, but also allowing an attacker to run an offline decryption attack with unlimited attempts. This invariably leads to your main password getting compromised.

Instead, set up SSH certificates, MFA, Yubikey, or TPM/Enclave storage for your private keys.

> You probably use an easy-to-remember password on SSH keys since you have to type them often

No, use ssh-agent and decrypt once per boot.

> Instead, set up SSH certificates, MFA, Yubikey, or TPM/Enclave storage for your private keys.

Granted, I agree with this, too.

> but also allowing an attacker to run an offline decryption attack with unlimited attempts. This invariably leads to your main password getting compromised.

Do the OpenSSH authors not know about PKBDF2 or similar?

How does PBKDF2 prevent an offline decryption attack with unlimited attempts?

All it does is slow down the attempts, but for the average person's easy-to-remember password, it's probably increasing the effort from milliseconds to a few days.

I always aimed for 15+ letter passwords and set at least 100 rounds of the key function? (The -a flag) when generating password protected ssh keys.