Right now, bots are scanning the internet for SSH servers. They’re testing default credentials against port 22 on every IP address they can find. If your SSH configuration is default, you’re in a lottery you don’t want to win.

This isn’t theoretical—it’s measurable. Set up a server with default SSH and watch the logs. Within hours, you’ll see login attempts from IP addresses around the world. They’re not targeting you specifically; they’re targeting everyone.

The good news: securing SSH isn’t complicated. A handful of configuration changes transforms an open door into a locked vault. The problem is nobody teaches this properly. You either get 50-page hardening guides that feel like overkill for a lab server, or vague advice like “use keys” without explaining why or how.

This guide bridges that gap. You’ll take a default SSH installation from vulnerable to properly secured, step by step. Not paranoid-security-researcher levels of hardening—practical security that stops real attacks without breaking your workflow.

If you’ve been putting off SSH security because it seemed complicated, this is your starting point. If you’ve been running default SSH configs because “nothing bad has happened yet,” consider this your wake-up call.

Why SSH Security Actually Matters

Let’s be clear about what’s happening: automated attacks against SSH are constant. Security researchers estimate millions of brute-force attempts happen daily across the internet. Default SSH configurations are targeted because they work often enough to be profitable.

Here’s what a vulnerable SSH setup looks like from an attacker’s perspective:

  • Password authentication enabled: Bots can try thousands of passwords per hour
  • Root login allowed: If they crack the password, they own the entire system
  • Default port 22: Easy to find during automated scans
  • No rate limiting: Unlimited guessing attempts

One compromised server leads to lateral movement through your network. Attackers install cryptominers, ransomware, or use your infrastructure to attack others. The cleanup costs far exceed prevention.

But SSH security isn’t just about external threats. Within organizations, proper SSH configuration means:

  • Audit trails: Key-based authentication ties actions to specific individuals
  • Access control: Limit who can SSH and from where
  • Compliance: Many frameworks (PCI-DSS, HIPAA, SOC 2) require SSH hardening
  • Professionalism: Default configs signal amateur administration

If you’re pursuing cybersecurity certifications or working toward roles in cloud infrastructure, SSH security knowledge is non-negotiable.

For system administrators moving beyond help desk, SSH security is a baseline expectation. Senior admins will judge your competence by how you configure server access.

Phase 1: Understanding What You’re Securing

Before changing settings, understand what SSH actually does and where the configuration lives.

The SSH Handshake (What Happens When You Connect)

When you type ssh user@server:

  1. Your client contacts the server on port 22 (by default)
  2. They negotiate encryption algorithms
  3. The server proves its identity (host key verification)
  4. You prove your identity (password or key)
  5. An encrypted tunnel is established
  6. Commands flow through the tunnel

Security vulnerabilities exist at step 4 (authentication) and in the configuration options that control what’s allowed.

Key Files and Locations

On the server (what you’re connecting to):

FilePurpose
/etc/ssh/sshd_configMain server configuration
/etc/ssh/ssh_host_*Server’s identity keys
~/.ssh/authorized_keysAllowed public keys per user

On the client (your machine):

FilePurpose
~/.ssh/configClient connection settings
~/.ssh/id_ed25519Your private key (keep secret)
~/.ssh/id_ed25519.pubYour public key (share freely)
~/.ssh/known_hostsServer keys you’ve verified

The /etc/ssh/sshd_config file is where server hardening happens. Every change in this guide modifies that file. If you’re still building Linux fundamentals, make sure you’re comfortable with file editing and permissions before diving into SSH configuration.

Checking Your Current Configuration

Before hardening, know what you’re starting with:

# View current sshd settings
sudo sshd -T | head -50

# Check if SSH is running
sudo systemctl status sshd

# See who's connected right now
who

# View recent SSH login attempts
sudo journalctl -u sshd | tail -50

On most systems, you’ll see password authentication enabled, root login permitted, and default port 22. These are the gaps we’re closing.

Phase 2: Generating and Deploying SSH Keys

Key-based authentication is the foundation of SSH security. Passwords can be guessed; properly generated keys cannot be brute-forced with current technology.

Why Keys Beat Passwords

A password like Tr0ub4dor&3 (12 characters, mixed case, numbers, symbols) has roughly 78 bits of entropy. An attacker with GPU hardware can test billions of combinations per second.

An Ed25519 SSH key has 256 bits of cryptographic security. Cracking it would require more energy than the sun will produce in its lifetime. The math is firmly on your side.

Beyond security, keys enable:

  • Passwordless login: More convenient, not less secure
  • Automation: Scripts can authenticate without storing passwords
  • Per-machine access: Revoke a stolen laptop without changing anything else
  • Passphrase protection: Add a password to your key for two-factor security

Generating a Modern SSH Key

Use Ed25519 for new keys. It’s faster, more secure, and creates smaller keys than RSA:

ssh-keygen -t ed25519 -C "[email protected]"

The -C flag adds a comment (typically your email) to identify the key.

When prompted:

  • File location: Press enter for the default (~/.ssh/id_ed25519)
  • Passphrase: Enter a strong passphrase to protect the key

If you must support older systems that don’t recognize Ed25519:

ssh-keygen -t rsa -b 4096 -C "[email protected]"

This generates a 4096-bit RSA key, which provides equivalent security but larger file sizes.

Deploying Your Public Key

Your public key needs to land in ~/.ssh/authorized_keys on the server. The secure way:

ssh-copy-id user@server

This command:

  1. Connects using your current authentication method
  2. Creates ~/.ssh/ on the server if needed
  3. Appends your public key to authorized_keys
  4. Sets correct permissions

If ssh-copy-id isn’t available, do it manually:

# Copy your public key content
cat ~/.ssh/id_ed25519.pub

# On the server, as the target user:
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "paste-your-public-key-here" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

File Permissions Matter

SSH refuses to use keys with wrong permissions. This is intentional—loose permissions mean others could read your private key.

# On your client machine
chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519      # Private key: owner only
chmod 644 ~/.ssh/id_ed25519.pub  # Public key: readable

# On the server
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

If SSH complains about permissions, these commands fix it. Understanding file permissions is essential—SSH is particularly strict about who can read key files.

Testing Key Authentication

Before disabling password authentication, verify keys work:

ssh -v user@server

The -v flag shows verbose output. Look for:

debug1: Authentications that can continue: publickey,password
debug1: Offering public key: /home/you/.ssh/id_ed25519
debug1: Server accepts key: /home/you/.ssh/id_ed25519
debug1: Authentication succeeded (publickey).

If you see Authentication succeeded (publickey), your key is working. If SSH falls back to password, something’s wrong with the key deployment.

Phase 3: Hardening sshd_config

Now that keys work, configure the server to enforce security. Edit /etc/ssh/sshd_config with:

sudo nano /etc/ssh/sshd_config

Or use vim if that’s your preference. Make these changes:

Disable Password Authentication

The most important change. Once key authentication works, disable passwords entirely:

PasswordAuthentication no

This stops brute-force attacks completely. Bots can guess passwords forever—they’ll never succeed.

Also disable these related options:

ChallengeResponseAuthentication no
UsePAM no

Disable Root Login

Never allow direct root SSH access:

PermitRootLogin no

This forces attackers to:

  1. Guess a valid username
  2. Compromise that user’s key
  3. Escalate privileges

If you need root access, SSH as a regular user and use sudo. For automation that requires root, use PermitRootLogin prohibit-password instead, which allows key-based root login but not passwords.

Enable Protocol 2 Only

SSH Protocol 1 has known vulnerabilities. Most modern systems default to Protocol 2, but explicitly enforce it:

Protocol 2

Limit User Access

If only specific users need SSH access:

AllowUsers alice bob charlie

Or restrict by group:

AllowGroups sshusers

Everyone not in the list is denied, regardless of valid credentials.

Set Idle Timeout

Disconnect inactive sessions automatically:

ClientAliveInterval 300
ClientAliveCountMax 2

This pings the client every 300 seconds (5 minutes). After 2 missed responses, the connection closes. Useful for preventing abandoned sessions.

Restrict IP Addresses (Optional)

If SSH access should only come from specific networks:

AllowUsers [email protected].*

Or use firewall rules (covered below) for more flexible control.

Complete Hardened Configuration

Here’s a full /etc/ssh/sshd_config with secure defaults:

# Basic settings
Port 22
Protocol 2
AddressFamily inet

# Authentication
PermitRootLogin no
PubkeyAuthentication yes
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

# Security
PermitEmptyPasswords no
X11Forwarding no
MaxAuthTries 3
MaxSessions 3

# Session settings
ClientAliveInterval 300
ClientAliveCountMax 2

# Logging
LogLevel VERBOSE

Applying Changes

After editing, test the configuration:

sudo sshd -t

If no errors appear, reload SSH:

sudo systemctl reload sshd

Use reload not restart to avoid dropping existing connections. Keep your current session open while testing from a new terminal—if something’s wrong, you can still fix it. This kind of careful change management is part of what separates entry-level IT from more senior roles.

Phase 4: Beyond sshd_config

Server configuration is the foundation, but several additional layers improve security.

Firewall Configuration

Rate-limit SSH connections to slow brute-force attempts. On Ubuntu/Debian with UFW:

sudo ufw limit 22/tcp
sudo ufw enable

The limit rule allows 6 connections in 30 seconds, then blocks for 30 seconds. Legitimate users rarely notice; attackers get significantly slowed.

On systems with iptables directly:

sudo iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set
sudo iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 30 --hitcount 6 -j DROP

For cloud servers, configure security groups in AWS, Azure, or Google Cloud to allow SSH only from known IPs if possible.

Fail2ban for Automatic Blocking

Fail2ban monitors logs and bans IPs that fail authentication repeatedly:

# Install
sudo apt install fail2ban

# Create local config
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit /etc/fail2ban/jail.local:

[sshd]
enabled = true
port = 22
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 3600
findtime = 600

This bans IPs for one hour after 3 failed attempts within 10 minutes. Start the service:

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Check banned IPs:

sudo fail2ban-client status sshd

Changing the Default Port (Controversial)

Changing SSH from port 22 to something else (like 2222) is “security through obscurity.” It won’t stop targeted attacks but does eliminate noise from automated scans.

In /etc/ssh/sshd_config:

Port 2222

Remember to:

  • Update firewall rules for the new port
  • Use -p 2222 in SSH commands
  • Update any scripts or automation

Opinions vary on whether this is worth the hassle. It reduces log noise significantly but doesn’t add real security against determined attackers.

SSH Keys for Service Accounts

For automated scripts that need SSH access:

  1. Create a dedicated service account
  2. Generate a key without a passphrase (or use ssh-agent)
  3. Restrict the key to specific commands

In authorized_keys, prefix the key with options:

command="/usr/local/bin/backup-script.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-ed25519 AAAA... service@backup

This key can only run that specific command. Even if compromised, it can’t do anything else.

Common SSH Problems and Fixes

When SSH stops working, systematic troubleshooting finds the problem quickly. The methodology mirrors general IT troubleshooting approaches—isolate the layer, test hypotheses, verify fixes.

”Permission denied (publickey)”

The server rejected your key. Causes:

  1. Key not in authorized_keys: Verify the public key is present
  2. Wrong permissions: Check .ssh (700) and authorized_keys (600)
  3. Wrong owner: Files must be owned by the connecting user
  4. SELinux blocking: Run restorecon -Rv ~/.ssh

Debug with verbose mode:

ssh -vvv user@server

Look for which key SSH tries and whether the server accepts it.

”Connection refused”

The SSH daemon isn’t listening. Check:

# Is sshd running?
sudo systemctl status sshd

# Is it listening on the right port?
sudo ss -tlnp | grep ssh

# Firewall blocking?
sudo iptables -L -n | grep 22

“Connection timed out”

Network issue or firewall. The server isn’t reachable:

# Can you reach the server at all?
ping server-ip

# Is port 22 reachable?
nc -zv server-ip 22

If ping works but port 22 times out, a firewall is blocking SSH traffic. Understanding networking fundamentals helps you diagnose these issues faster.

”Host key verification failed”

The server’s identity changed (or someone is intercepting your connection):

# View stored host key
ssh-keygen -F server-ip

# Remove old key if server was rebuilt
ssh-keygen -R server-ip

Only remove the old key if you’re certain the server changed legitimately (reinstall, new IP, etc.).

Locked Out After Configuration

This is why you keep an existing session open while testing. If you’re locked out:

  1. Console access: Use cloud provider’s console, KVM, or physical access
  2. Recovery mode: Boot into single-user mode and fix /etc/ssh/sshd_config
  3. Backup access: Some providers offer emergency rescue systems

Prevention: Always test SSH changes from a second terminal while one session remains connected.

SSH Client Configuration

Beyond server hardening, your client-side configuration affects security and convenience. Mastering these settings is part of becoming proficient with Bash and the command line.

The ~/.ssh/config File

Instead of typing full SSH commands, define hosts in ~/.ssh/config:

Host web-prod
    HostName 203.0.113.50
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Host bastion
    HostName jump.example.com
    User admin

Host internal-*
    ProxyJump bastion

Now ssh web-prod connects with all those options automatically. The ProxyJump directive routes through a bastion host—useful for reaching servers that aren’t directly accessible.

SSH Agent for Key Management

If your key has a passphrase (it should), typing it repeatedly is tedious. SSH agent caches the decrypted key:

# Start agent
eval "$(ssh-agent -s)"

# Add key (enter passphrase once)
ssh-add ~/.ssh/id_ed25519

# List cached keys
ssh-add -l

On macOS, add to ~/.ssh/config:

Host *
    AddKeysToAgent yes
    UseKeychain yes

This integrates with the macOS keychain for persistent caching.

Agent Forwarding (Use Carefully)

Agent forwarding lets you use your local keys on remote servers:

ssh -A user@bastion
# Now you can SSH from bastion using your local keys

Convenient but risky: anyone with root on the intermediate server can hijack your agent. Only use agent forwarding on servers you fully trust.

SSH for Automation and DevOps

Proper SSH configuration enables secure automation across infrastructure.

Non-Interactive Authentication

For scripts, either:

  1. Passphrase-less key: Generate without a passphrase for fully automated access
  2. SSH agent: Start an agent and add the key before the script runs
  3. ssh-keyscan: Preload host keys to avoid interactive prompts
# Add host key to known_hosts non-interactively
ssh-keyscan -H server >> ~/.ssh/known_hosts

Configuration Management Integration

Tools like Ansible use SSH for remote execution. Ensure:

  • SSH keys are deployed to all managed hosts
  • The control machine can reach all targets
  • host_key_checking = False in ansible.cfg for initial deployment (temporarily)

Proper SSH configuration makes DevOps automation possible. Without it, you’re stuck with manual server access.

Bastion/Jump Hosts

For servers not directly accessible from the internet:

Internet → Bastion → Internal Servers

Configure ProxyJump in ~/.ssh/config:

Host bastion
    HostName bastion.example.com
    User admin

Host *.internal
    ProxyJump bastion

Now ssh database.internal automatically routes through the bastion.

SSH Skills for Job Interviews

SSH configuration questions appear in almost every sysadmin interview. Be ready for:

“How would you secure an SSH server?”

Strong answer: “Disable password authentication and require key-based auth. Disable root login. Use AllowUsers to restrict who can connect. Implement fail2ban for automatic IP blocking. Consider non-standard port to reduce log noise.”

“Walk me through SSH key authentication.”

Strong answer: “User generates a key pair—public and private. Public key goes in authorized_keys on the server. When connecting, the client proves it has the private key through cryptographic challenge-response. The private key never leaves the client.”

“You can’t SSH to a server. How do you troubleshoot?”

Strong answer: “First, verify network connectivity with ping. Then check if port 22 is reachable with netcat. Use ssh -vvv for verbose debugging. Check if sshd is running and listening. Review /var/log/auth.log for authentication issues. Verify key permissions.”

These questions test practical knowledge, not memorization. Practice troubleshooting real SSH issues in your home lab to build intuition.

Next Steps After Securing SSH

With SSH hardened, you have a foundation for broader security work:

Monitor access continuously: Set up alerts for SSH logins, especially from unexpected IPs. Tools like OSSEC or centralized logging catch suspicious activity.

Implement certificate-based SSH: For larger deployments, SSH certificates simplify key management. Instead of distributing public keys to every server, a central authority signs short-lived certificates.

Add two-factor authentication: Google Authenticator or hardware keys (YubiKey) add another layer for sensitive systems. This is especially relevant if you’re working toward Security+ or other security certifications.

Practice command-line security skills: SSH is just one piece. Building broader Linux security skills through platforms like Shell Samurai reinforces these concepts with hands-on practice.

The configuration you’ve implemented stops the vast majority of attacks. Each additional layer reduces risk further but adds complexity. Balance security needs with operational reality.

SSH skills directly map to IT certifications like Network+, Security+, and Linux+ that validate your server administration abilities.

Quick Reference: SSH Hardening Checklist

Use this checklist when securing a new server:

Key Setup

  • Generate Ed25519 key: ssh-keygen -t ed25519
  • Deploy key with ssh-copy-id user@server
  • Verify key auth works before disabling passwords
  • Set permissions: .ssh (700), authorized_keys (600)

Server Configuration

  • PasswordAuthentication no
  • PermitRootLogin no
  • Protocol 2
  • MaxAuthTries 3
  • ClientAliveInterval 300
  • Test config: sudo sshd -t
  • Reload: sudo systemctl reload sshd

Additional Layers

  • Firewall rate limiting: ufw limit 22/tcp
  • Install and configure fail2ban
  • Set up log monitoring/alerting

Client Configuration

  • Create ~/.ssh/config for common hosts
  • Set up SSH agent for passphrase caching

FAQ

Is changing the SSH port worth the hassle?

It depends on your threat model. Changing ports eliminates most automated scanning noise, which makes logs cleaner and reduces CPU load from rejected connections. Against targeted attacks, it provides zero additional security—attackers will find the port. For internet-facing servers that attract constant automated probes, the reduced noise may be worth the minor inconvenience. For internal servers, stick with port 22 and focus on real security measures like key-only authentication.

Can I use the same SSH key on multiple servers?

Yes, and this is standard practice. Your public key can be in authorized_keys on as many servers as you need to access. The security model assumes your private key stays private on your machine. Just don’t deploy the same key across machines you don’t control—generate separate keys for work and personal use, or for machines with different trust levels.

What happens if I lose my SSH private key?

You lose access to any server that only accepts that key. This is why keeping backup access methods matters during initial setup. Options: physical console access, cloud provider’s web console, recovery mode, or deploying multiple keys before disabling passwords. For critical infrastructure, always have an out-of-band access method that doesn’t depend on SSH.

How often should I rotate SSH keys?

Annual rotation is reasonable for most environments. More frequent rotation (quarterly) is appropriate for highly sensitive systems or compliance requirements. The bigger risk isn’t key age—it’s key sprawl (keys you’ve forgotten about on systems you no longer manage). Audit where your public key lives periodically and remove it from systems you no longer need to access.

Should I use SSH agent forwarding?

Be cautious. Agent forwarding lets you use your local keys from a remote server—convenient for hopping through bastion hosts. But anyone with root access on the intermediate server can hijack your forwarded agent to access other systems. Only enable forwarding to servers you completely trust, and prefer ProxyJump (which doesn’t expose your agent) for routine bastion access.