Managing SSH Keys, Passwords, and Dev Secrets on Linux

Every Linux machine I use slowly collects credentials: SSH keys for servers and GitHub, a password for that one old box that won't take a key, a database password in a .env file, an npm token, an API key I pasted into a terminal "just for a minute." None of it feels dangerous on its own. Together it is basically the keys to everything I work on, often sitting in plain text in my home directory.

This post is how I keep that under control. It covers three things: SSH keys, the other passwords the system needs (including SSH hosts that only accept a password), and secrets for development like API keys and tokens. Everything here uses standard, open source tools that are in the repos of every major distro.

If you want the broader list of security software I use, see Linux Security Tools. This post is the hands-on companion to it.

TL;DR

• One SSH key per device, Ed25519, protected with a passphrase. Let an agent hold it unlocked in memory so you type the passphrase once, not every connection.
• Use ~/.ssh/config so each host uses the right key and nothing else (IdentitiesOnly yes).
• Every other password goes in a password manager (pass, KeePassXC, or your desktop keyring). Scripts fetch passwords from it; they never contain them.
• Never put a secret on the command line where it lands in shell history and ps output. That includes sshpass -p.
• Dev secrets stay out of git. Use .env files that are gitignored, or encrypt them with SOPS + age if they need to be committed. Run a leak scanner like gitleaks before you push.
• If a secret leaks, rotate it first. Deleting the commit doesn't make the secret safe again.
• Back up your keys and password database, encrypted, somewhere other than the laptop.

Part 1: SSH keys

Generate a good key

Ed25519 is the right default today. Keys are small and fast, and every OpenSSH server from the last decade supports them.

ssh-keygen -t ed25519 -a 100 -C "you@laptop-2026"

• -a 100 sets the number of KDF rounds used to protect the private key with your passphrase. More rounds make brute-forcing a stolen key file much slower.
• -C is just a comment, but make it say which device the key lives on. When you are cleaning up authorized_keys two years from now, "you@laptop-2026" is a lot more useful than "you@localhost."
• Always set a passphrase. Without one, anyone who copies the file (a backup, a stolen disk, a malicious package running as your user) has the key.

Only fall back to RSA if a server or appliance really can't do Ed25519, and if you do, use at least 3072 bits:

ssh-keygen -t rsa -b 4096 -a 100 -C "you@laptop-2026-legacy" -f ~/.ssh/id_rsa_legacy

I use one key per device rather than copying one key everywhere. If a laptop is lost, I revoke that one key and nothing else breaks. I also keep separate keys for very different purposes (personal servers vs. work vs. GitHub), so one leaking doesn't open everything.

Permissions matter

OpenSSH refuses to use a private key that other users can read, and it's good hygiene anyway:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519 ~/.ssh/config
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/authorized_keys   # on servers

The files you'll actually deal with:

Put the key on a server

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server.example.com

That appends your public key to the server's authorized_keys and fixes the permissions. Once key login works, consider turning password login off on servers you control by setting PasswordAuthentication no in /etc/ssh/sshd_config (or a file in /etc/ssh/sshd_config.d/) and reloading sshd. Keep a second session open while you test so you can't lock yourself out.

Use ~/.ssh/config

This file is the single biggest quality-of-life improvement for SSH. It lets you give hosts short names and pin each one to the right key:

# Defaults for every host
Host *
    AddKeysToAgent yes
    IdentitiesOnly yes
    HashKnownHosts yes
    ServerAliveInterval 60

Host github.com
    User git
    IdentityFile ~/.ssh/id_ed25519_github

Host homelab
    HostName 192.168.1.20
    User me
    IdentityFile ~/.ssh/id_ed25519

Host work-*
    User me.work
    IdentityFile ~/.ssh/id_ed25519_work

• IdentitiesOnly yes makes SSH offer only the key you listed, instead of trying every key in your agent. That avoids "too many authentication failures" errors, and it means a server doesn't learn about every key you own.
• AddKeysToAgent yes adds the key to your agent the first time you use it, so you only type the passphrase once. You can give it a lifetime instead, e.g. AddKeysToAgent 4h.
• HashKnownHosts yes stores hostnames in known_hosts as hashes, so someone who reads the file doesn't get a list of every server you use. To hash an existing file, run ssh-keygen -H (it leaves a known_hosts.old backup you should delete).

Now ssh homelab just works. To see what SSH will actually use for a host, ssh -G homelab prints the final, merged configuration.

Let an agent hold your keys

A passphrase is only bearable if you don't type it fifty times a day. An SSH agent keeps the decrypted key in memory for your session.

• Desktop sessions usually have one already. GNOME runs gcr-ssh-agent (it used to be part of gnome-keyring), and KDE works with ssh-agent plus ksshaskpass so the passphrase can be stored in KWallet. Check with echo $SSH_AUTH_SOCK.
• Plain terminal / servers / WSL: start one yourself, or use keychain, which reuses a single agent across all your shells:

# in ~/.bashrc or ~/.zshrc
eval "$(keychain --eval --quiet id_ed25519)"

• KeePassXC has a built-in SSH agent integration (Settings → SSH Agent). Keys stored in the database are added to the agent when you unlock it and removed when it locks. That's a nice way to tie "my laptop is unlocked" to "my keys are usable."

Useful agent commands:

ssh-add -l                         # list loaded keys
ssh-add -t 4h ~/.ssh/id_ed25519    # load a key that expires after 4 hours
ssh-add -D                         # remove all keys (e.g. before walking away)

Be careful with agent forwarding (ForwardAgent yes / ssh -A). Anyone with root on the remote host can use your forwarded agent while you're connected. If you need to hop through a bastion host, use ProxyJump instead; it doesn't expose your agent:

Host internal-db
    HostName 10.0.0.15
    ProxyJump bastion

Change passphrases and rotate keys

ssh-keygen -p -f ~/.ssh/id_ed25519   # change the passphrase, key stays the same

To rotate a key, generate a new one, add the new public key everywhere (servers, GitHub, GitLab), confirm it works, and then remove the old public key from each authorized_keys and from your Git hosting account. Rotating when you replace a laptop is a good habit, and the per-device comments make it obvious which lines to delete.

Hardware keys (FIDO2) — optional but nice

If you have a YubiKey, Nitrokey, SoloKey or similar FIDO2 device, OpenSSH (8.2+) can use it directly. The private key never leaves the hardware, and each login needs a physical touch:

ssh-keygen -t ed25519-sk -O resident -O verify-required -C "you@yubikey"

-O resident stores the key handle on the device so you can recreate the files on a new machine with ssh-keygen -K. -O verify-required also asks for the device PIN. The server just needs a reasonably modern OpenSSH. Buy two keys and register both, so losing one isn't a lockout.

Part 2: Passwords used elsewhere on the system

SSH keys cover a lot, but a Linux system still needs plenty of ordinary passwords: an old SSH host or network device that only does password login, a database, a mail account, a VPN, a web service a script talks to. The rule for all of them is the same: they live in a password manager, and everything else asks the password manager.

Pick a password manager that has a CLI

pass in two minutes:

pass init YOUR_GPG_KEY_ID        # use an existing GPG key
pass git init                    # optional: track the store with git
pass insert servers/oldbox       # prompts for the password, nothing on the command line
pass generate web/someservice 24 # create a random 24-character password
pass show servers/oldbox         # print it
pass -c servers/oldbox           # copy to clipboard, cleared after 45 seconds

KeePassXC from a script:

keepassxc-cli show -s -a Password ~/Passwords.kdbx "servers/oldbox"

Desktop keyring with secret-tool (from the libsecret tools package):

secret-tool store --label="oldbox ssh" service ssh host oldbox   # prompts for the secret
secret-tool lookup service ssh host oldbox

KeePassXC can also act as the Secret Service provider (Settings → Secret Service Integration), so apps and secret-tool that expect GNOME Keyring end up reading from your KeePassXC database instead. Only run one Secret Service provider at a time.

SSH hosts that only take a password

The best fix is to make it take a key. Even old devices usually support authorized_keys. If you really can't:

1. Type it less often with connection sharing. SSH can reuse one authenticated connection for later sessions, scp and rsync, so you enter the password once:

Host oldbox
    HostName 192.168.1.50
    User admin
    PubkeyAuthentication no
    ControlMaster auto
    ControlPath ~/.ssh/cm-%C
    ControlPersist 10m

2. If you must automate it, never use sshpass -p. sshpass -p 'hunter2' ssh oldbox puts the password in your shell history and in the process list, where any user on the machine can see it with ps. Feed it from your password manager instead:

# Read the password from the SSHPASS environment variable, set only for this command
SSHPASS="$(pass show servers/oldbox)" sshpass -e ssh oldbox

# Or read it from a file descriptor, so it's never in the environment at all
sshpass -f <(pass show servers/oldbox) ssh oldbox

This is still weaker than a key, because the password exists in a script's environment. Treat it as a stopgap.

Other places passwords hide

Git over HTTPS. Don't use credential.helper store; it writes your token to ~/.git-credentials in plain text. Use one of these instead:

git config --global credential.helper 'cache --timeout=3600'   # memory only, for an hour
git config --global credential.helper libsecret                 # desktop keyring
gh auth setup-git                                               # let the GitHub CLI handle it

The libsecret helper ships with git on Arch and Fedora (git-credential-libsecret). On Debian/Ubuntu you build it from /usr/share/doc/git/contrib/credential/libsecret. Better yet, use SSH for Git and skip HTTPS passwords entirely.

Dotfiles that legitimately hold passwords like ~/.netrc, ~/.pgpass (PostgreSQL) and ~/.my.cnf (MySQL) should be chmod 600. Many tools refuse to use them otherwise. If a tool can call a command for its password (many can, e.g. passwordeval in msmtp or PassCmd in isync), point it at pass show ... instead of storing the password at all.

Shell history. Secrets end up in ~/.bash_history and ~/.zsh_history more often than anywhere else. Set your shell to skip lines that start with a space, and get in the habit of prefixing anything sensitive with one:

# bash (~/.bashrc)
HISTCONTROL=ignoreboth

# zsh (~/.zshrc)
setopt HIST_IGNORE_SPACE

Better still, read secrets interactively so they never appear on the command line: read -rs DB_PASS prompts without echoing.

Part 3: Secrets for development

API keys, database URLs, OAuth client secrets, cloud credentials, and package registry tokens are where I've seen the most real leaks. They get copied into code "temporarily," committed, pushed, and scraped by bots within minutes of hitting a public repo.

The ground rules

1. Secrets never go in source code or git history. Not in config files, not in tests, not in a commented-out line.
2. Each person and each environment gets its own secret. Your laptop's dev key should not be the production key.
3. Least privilege, short lifetimes. Use scoped tokens (read-only, one repo, one bucket) and expiration dates wherever the service allows them.
4. Assume you'll have to rotate every secret someday, and make that easy.

Local .env files, done properly

.env files are fine for local development as long as they stay local:

echo ".env" >> .gitignore
echo ".env.*" >> .gitignore
echo "!.env.example" >> .gitignore
chmod 600 .env

Commit a .env.example with placeholder values so others know what's needed:

# .env.example
DATABASE_URL=postgres://user:password@localhost:5432/myapp_dev
STRIPE_API_KEY=sk_test_replace_me

Even better, don't keep the real values in .env at all. direnv loads environment variables when you cd into a project and unloads them when you leave, and an .envrc can pull values straight from your password manager:

# .envrc (gitignored)
export DATABASE_URL="$(pass show dev/myapp/database_url)"
export STRIPE_API_KEY="$(pass show dev/myapp/stripe_test_key)"

Run direnv allow once. Now the secrets exist only in your password store and in memory while you're in that directory.

For a one-off command, scope the secret to just that process:

STRIPE_API_KEY="$(pass show dev/myapp/stripe_test_key)" npm run dev

Encrypted secrets you can commit: SOPS + age

Sometimes a team really does want secrets in the repo, for example shared dev credentials or deployment config. SOPS with age keys encrypts the values in a YAML, JSON, or .env file while leaving the keys readable, so diffs still make sense.

age-keygen -o ~/.config/sops/age/keys.txt   # prints your public key (age1...)

# .sops.yaml (committed)
creation_rules:
  - path_regex: secrets/.*
    age: age1yourpublickeyhere,age1teammatepublickey

sops -e -i secrets/dev.env                 # encrypt in place
sops secrets/dev.env                       # open decrypted in $EDITOR, re-encrypt on save
sops exec-env secrets/dev.env 'npm run dev' # run with the values as env vars

Your age private key in ~/.config/sops/age/keys.txt now becomes something to back up and protect like an SSH key. git-crypt is an alternative that transparently encrypts whole files using GPG.

Containers: don't bake secrets into images

ENV and ARG values in a Dockerfile end up in the image metadata and layers. Anyone who can pull the image can read them with docker history or docker inspect. Use build secrets instead:

# Dockerfile
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci

docker build --secret id=npmrc,src=$HOME/.npmrc .

For running containers, Compose secrets: mounts each secret as a file under /run/secrets/ instead of an environment variable:

services:
  app:
    image: myapp
    secrets:
      - db_password
secrets:
  db_password:
    file: ./secrets/db_password.txt   # gitignored

CI/CD

Use your CI platform's secret store (GitHub Actions secrets and environments, GitLab CI variables marked masked and protected) rather than committing anything. For cloud access, prefer OIDC federation (GitHub Actions can request short-lived AWS, GCP, or Azure credentials with permissions: id-token: write) over long-lived access keys stored as secrets. Then there's no permanent key to leak.

Package registry and CLI tokens

• npm: use granular access tokens with an expiry, and reference them from ~/.npmrc through an environment variable instead of pasting them in:

  ini

  //registry.npmjs.org/:_authToken=${NPM_TOKEN}

• PyPI: use project-scoped API tokens, and chmod 600 ~/.pypirc (or better, use trusted publishing from CI).

• GitHub CLI: gh auth login stores its token in the system keyring when one is available. Check with gh auth status.

• Cloud CLIs (~/.aws/credentials, ~/.config/gcloud, ~/.azure): prefer SSO / short-lived login (aws sso login, gcloud auth login) over static access keys sitting in a file.

Catch leaks before they're pushed

gitleaks scans for things that look like keys and tokens. Run it as a pre-commit hook so a secret never makes it into a commit:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/gitleaks/gitleaks
    rev: v8.28.0   # pin to the latest release
    hooks:
      - id: gitleaks

Or scan by hand:

gitleaks git -v     # scan the repo's full history
gitleaks dir .      # scan the working directory, including untracked files

Also turn on secret scanning and push protection on GitHub (it's free for public repos), which blocks pushes containing known token formats.

If a secret leaks anyway

1. Revoke or rotate the secret immediately. This is the step that actually fixes it. Bots scan public repos constantly, so assume it was copied the moment it was pushed.
2. Check the service's logs for use you don't recognize.
3. Then clean it out of history if you want to, with git filter-repo (for example git filter-repo --replace-text expressions.txt), and force-push. Removing it from history is housekeeping, not a fix. Forks, clones, and caches may already have it.

Backups and recovery

Good key hygiene can lock you out if you're not careful. Things to back up, encrypted, somewhere other than the laptop:

• ~/.ssh (private keys are passphrase-protected, but still keep the backup encrypted)
• Your password store: the pass git repo (pushed to a private remote; it's GPG-encrypted) or your KeePassXC .kdbx file, synced to a second device
• Your GPG private key (if you use pass) and your age key (if you use SOPS). Without them, those backups are unreadable.
• A written copy of your password manager's master password and your GPG key's passphrase, stored somewhere physically safe

For backing up, rsync to an encrypted drive, or an encrypted backup tool like restic or Borg, works well.

If a laptop is lost or stolen:

1. Remove that device's public key from every server's authorized_keys and from GitHub/GitLab. (This is where one-key-per-device pays off.)
2. Revoke tokens that lived on it: gh, npm, cloud CLIs, anything in .env files.
3. Change the master password of your password manager if the database was on the device, and then the most important passwords inside it.

A quick audit you can run today

ls -la ~/.ssh                              # permissions: 700 on the directory, 600 on private keys
ssh-add -l                                 # what's loaded in your agent right now
for k in ~/.ssh/*.pub; do ssh-keygen -lf "$k"; done   # key types and sizes; retire old RSA/DSA keys
grep -n "credential.helper" ~/.gitconfig   # make sure it isn't "store"
ls -la ~/.git-credentials 2>/dev/null      # if this exists, it's plaintext
grep -rIl -i "password\|token\|secret" ~/.config ~/.*rc 2>/dev/null   # rough check for stray secrets
gitleaks dir .                             # in each project directory

None of this is exotic. It's mostly about deciding, once, where secrets are allowed to live (an agent, a password manager, an encrypted file) and then making the easy path the safe one, so you're never tempted to paste a password into a script "just this once."