Linux server hardening: a baseline for the first hour of a new server

Assumptions and when this baseline does not apply

This is a baseline for a general-purpose server you administer yourself: a fresh install of a currently supported Ubuntu LTS, Debian stable, or RHEL-family release (RHEL, AlmaLinux, Rocky Linux), on a VPS or bare metal, reachable over SSH. Run every command as a user with sudo. Commands are shown for both families where they differ.

It is not enough on its own, or not the right tool, when:

• You must meet a formal standard. Use the full CIS Benchmark for your distribution and version, or your auditor's control set. This guide covers the high-value subset, not every control.
• The server is managed by someone else. On managed hosting, the provider may own patching and SSH configuration; changing them yourself can break their tooling. Agree who does what first (see shared responsibility).
• You run containers or Kubernetes nodes built from immutable images. Harden the image and the orchestration layer instead of each node by hand.
• The host is a firewall, VPN or appliance with its own vendor configuration.

Where the distributions differ

Leave SELinux in enforcing mode and AppArmor enabled. When an application fails because of them, fix the policy or label; switching them off removes a layer that stops many real exploits from going further.

Users and sudo

Give every person their own named account and use sudo for admin work. Shared accounts and direct root logins make it impossible to tell from the logs who did what.

# Debian / Ubuntu
sudo adduser alice
sudo usermod -aG sudo alice

# RHEL family
sudo useradd -m alice
sudo passwd alice
sudo usermod -aG wheel alice

Keep a password on the account for sudo, even though SSH will use keys only. Edit sudo rules only with visudo (or a file under /etc/sudoers.d/ checked with visudo -c), because a syntax error in sudoers can remove sudo access entirely. Avoid NOPASSWD: ALL for human accounts; if an automation account needs passwordless sudo, limit it to the specific commands it runs.

SSH keys and sshd_config

Create a key on your own workstation, not on the server, and copy the public half up. Ed25519 keys are short, fast and supported by every current OpenSSH release.

# On your workstation
ssh-keygen -t ed25519 -C "alice@workstation"
ssh-copy-id alice@server.example.com

# Test in a NEW terminal before changing anything else
ssh alice@server.example.com

Put your settings in a drop-in file rather than editing the main file. Current Debian, Ubuntu and RHEL-family releases include /etc/ssh/sshd_config.d/*.conf near the top of sshd_config, and for each keyword sshd uses the first value it reads. Some images ship a drop-in (for example from cloud-init) that turns password login back on, so give yours a name that sorts first, such as 00-hardening.conf.

# /etc/ssh/sshd_config.d/00-hardening.conf
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
MaxAuthTries 3
LoginGraceTime 30
X11Forwarding no
AllowGroups sshusers
ClientAliveInterval 300
ClientAliveCountMax 2

sudo groupadd sshusers
sudo usermod -aG sshusers alice

# Check syntax, then show the effective values
sudo sshd -t
sudo sshd -T | grep -Ei 'permitrootlogin|passwordauthentication|authenticationmethods|allowgroups'

# Apply: Debian/Ubuntu
sudo systemctl reload ssh
# Apply: RHEL family
sudo systemctl reload sshd

Now open a third session from a new terminal. Only when it logs in with your key should you close the original one. On recent Ubuntu releases sshd is socket-activated: if you change Port or ListenAddress, you must also run systemctl daemon-reload and restart ssh.socket, as Ubuntu's release notes describe. Changing the port reduces log noise from scanners but is not a security control on its own.

Firewall: default deny inbound

Allow SSH first, then your service ports, then turn on default deny. Doing it in the other order is the most common way to lock yourself out.

# Ubuntu with ufw
sudo ufw allow OpenSSH
sudo ufw allow 443/tcp
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw enable
sudo ufw status verbose

# RHEL family with firewalld
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

If you prefer nftables directly, keep the ruleset in /etc/nftables.conf, check it with nft -c -f before loading, and include rules for established connections and loopback before the final drop. Where you can, restrict SSH to known source addresses (an office range or a VPN) instead of the whole internet.

If your provider offers a network firewall in front of the server, use it as well. Two layers mean one mistake does not expose a service.

Automatic security updates and reboots

NIST SP 800-40 treats patching as routine maintenance rather than an emergency process, and most servers are compromised through known, already-fixed vulnerabilities. Turn on automatic installation of security updates and decide how reboots will happen.

# Debian / Ubuntu
sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
# Is a reboot pending?
ls /var/run/reboot-required 2>/dev/null

# RHEL family
sudo dnf install dnf-automatic
# In /etc/dnf/automatic.conf set:  upgrade_type = security  and  apply_updates = yes
sudo systemctl enable --now dnf-automatic.timer
# Is a reboot pending?
sudo dnf needs-restarting -r

Kernel and C library updates only take effect after a reboot. Either allow automatic reboots in a quiet window (unattended-upgrades has an Automatic-Reboot option) or schedule them yourself and alert when a reboot has been pending for more than a few days. A patched package that is not yet running is not patched.

Time sync, logging and retention

Accurate time matters for TLS, for authentication tokens and for matching log lines across servers during an incident. Check that one time service is active and synchronized:

timedatectl status        # look for "System clock synchronized: yes"
chronyc tracking          # if chrony is in use

Make the systemd journal persistent so logs survive a reboot, and cap its size:

# /etc/systemd/journald.conf.d/00-retention.conf
[Journal]
Storage=persistent
SystemMaxUse=1G
MaxRetentionSec=90day

sudo systemctl restart systemd-journald
journalctl --disk-usage

Local logs are the first thing an intruder deletes. Forward authentication and system logs to a separate host or service the server cannot modify, and decide retention by what you would need in an investigation; NIST SP 800-92 covers log management planning. On RHEL-family systems, auditd is installed by default and records privileged actions; keep it running.

With password SSH disabled, brute-force attempts cannot succeed, but they fill logs. fail2ban (packaged in Debian and Ubuntu, and in EPEL for RHEL-family systems) bans addresses after repeated failures; enable its sshd jail and any jail matching your web server. Rate-limiting new SSH connections in nftables is an equivalent approach.

Minimal services and file permissions

Everything listening on a port is attack surface. List what listens and what starts at boot, then remove what you do not need:

sudo ss -tulpn
systemctl list-unit-files --type=service --state=enabled

Databases, caches and admin panels should listen on 127.0.0.1 or a private network, not on all interfaces. Then check the permissions that matter most:

chmod 700 ~/.ssh && chmod 600 ~/.ssh/authorized_keys
# World-writable files on the root filesystem (review each result)
sudo find / -xdev -type f -perm -0002 -ls
# SUID/SGID binaries (compare against a fresh install)
sudo find / -xdev \( -perm -4000 -o -perm -2000 \) -type f -ls

Run each application under its own unprivileged user, owning only its own directories. Web roots should not be writable by the web server process except for specific upload directories.

Secrets, backups and monitoring

Secrets. Database passwords, API keys and TLS private keys should live in files readable only by the service user (mode 600) or in a secret manager, never in a git repository, a world-readable config or shell history. A leading space before a command keeps it out of bash history when HISTCONTROL includes ignorespace, but the safer habit is never to type secrets on the command line. Rotate any secret that has been in a repository, even briefly.

Backups. Hardening reduces the chance of compromise; backups decide how bad one is. Keep at least one copy off the server and outside the reach of its credentials, and test a restore. The backup and recovery guide covers RPO, RTO and the 3-2-1 rule.

Monitoring. At minimum, alert a human on: disk above 80 %, memory exhaustion, the main service down, certificate expiry within 21 days, a reboot pending for more than a week, and a spike in failed logins. An alert nobody receives is the same as no alert, so send a test.

Verification checklist

Run these after the baseline and again after any major change. Each item has a command or test that proves it, not just a setting you believe is applied.

• Root SSH login is refused: ssh root@server fails; sshd -T shows permitrootlogin no.
• Password login is refused: ssh -o PubkeyAuthentication=no alice@server fails with "Permission denied (publickey)".
• Only expected ports are open from outside: scan from another host, for example nmap -Pn server, and compare with ss -tulpn.
• Firewall is active with default deny: ufw status verbose or firewall-cmd --list-all.
• Automatic security updates are scheduled: systemctl list-timers shows the apt or dnf-automatic timers.
• No reboot has been pending for long: /var/run/reboot-required absent or dnf needs-restarting -r clean.
• Clock is synchronized: timedatectl says yes.
• Journal is persistent and logs reach the remote log host.
• SELinux is enforcing or AppArmor profiles are loaded.
• fail2ban (or equivalent) shows the sshd jail active: fail2ban-client status sshd.
• No secrets in world-readable files or in the application's git history.
• A backup restore has been tested on a separate machine.
• A test alert reached a person.

Common mistakes

• Disabling passwords before testing the key in a new session.
• Enabling the firewall before allowing SSH.
• Editing sshd_config while a cloud-init drop-in overrides it; sshd -T shows what actually applies.
• Assuming ufw protects Docker ports. It does not by default.
• Turning off SELinux to fix one application.
• Installing updates but never rebooting, so the old kernel keeps running.
• Keeping logs only on the server that might be compromised.
• Hardening once and never re-checking. Re-run the checklist after upgrades and new software.

Related