How to replace public SSH keys on your AWS EC2 instance

Security can be easily overlooked when building a product, especially when working with an outsourced engineering team. You want to trust them, so you give them access to your servers. But then you discover fraudulent activity, and, well, you start to panic.

In hindsight, you realize you never should have shared your Secure Shell (SSH) key, instead storing it in a vault with restricted user access. If, however, someone has a private SSH key to your Amazon Web Services (AWS) Elastic Compute Cloud (EC2) instance and you’re worried about a malicious attack, you have two options to revoke their access:

1. Create a new key-pair in the AWS console and boot up a new instance (assuming the attacker is removed from IAM users). This requires configuring the instance, which can be time-consuming — especially when you have several of them.
2. Replace the public key in ~/.ssh/authorized_keys on your existing instance so the attacker can no longer unlock it with their private key.

Here’s a summary of how to replace the keys mentioned in option No. 2 above:

(For more, DigitalOcean has a great tutorial on setting up SSH keys.)

1. On your local machine in the terminal, generate a new key pair: ssh-keygen -t rsa
2. When prompted to save the file, hit Enter for the default location or choose your own path.
3. When prompted for a passphrase, you can leave the field empty. Although it does not hurt to have more security, if the key pair is used elsewhere for CI or automation, you will need to leave the passphrase empty — machines cannot guess passphrases.
4. Copy the public key you just saved on your machine to your EC2 authorized keys file:
   cat ~/.ssh/id_rsa.pub | ssh user@123.45.56.78 "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys where ~/.ssh/id_rsa.pub is the new key on your machine and user@123.45.56.78 is the username and IP address of your EC2 instance.
5. At this point, your new public key should be on your EC2 instance in the authorized_keys file, and all you have to do is remove the old one. Make sure you can SSH into your EC2 instance with the new key first.
6. Once you’re in, you can remove the old key using vim ~/.ssh/authorized_keys Just go to the line with the old key and remove it: dd Note: If you tried editing the file and didn’t save it, or the connection was interrupted, an .authorized_keys.swp file will be created, and the next time you try to edit your authorized_keys, you will get a nasty message. Just delete the .swp file, and you should be good to edit.
7. Save the file.

Make sure to update the key if you’re using it elsewhere, like on a continuous integration (CI) server. Otherwise you’ll be scratching your head when none of your builds are working.