Troubleshoot Linux Disk Space and Inode Exhaustion

When a Linux filesystem fills, applications may fail to write logs, databases may stop, and package operations can break. Start by confirming which mount is affected, then distinguish exhausted space from exhausted inodes before deciding what to clean up.

The commands below are primarily diagnostic. Run them with appropriate privileges, and inspect ownership and retention requirements before removing files. Do not delete files from a live database, application, or system directory just because they are large.

---

Step 1: Identify the Full Mount

01

Check Filesystem Space and Inodes

Triage

Use human-readable output to find the affected mount and check inode consumption separately. A filesystem can report free gigabytes but still reject new files when its inodes are exhausted. The final command shows the mount containing the current directory; replace . with the path reported by the failing application if needed.

df -hT
df -ih
findmnt -T .

❯ View Expected Console Output

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/vda1      ext4   40G   38G  1.2G  97% /

Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/vda1      2.5M   1.1M  1.4M   45% /

---

Step 2: Find Which Top-Level Directory Is Growing

02

Measure Usage Without Crossing Mounts

Locate Usage

Use du on the affected mount to identify large directories. The -x option keeps the scan on one filesystem, so a separate mounted volume is not included. The first scan can take time on large filesystems; narrow it to the affected path when possible.

sudo du -xhd1 / 2>/dev/null | sort -h
sudo du -xhd1 /var 2>/dev/null | sort -h

❯ View Expected Console Output

120M    /etc
2.1G    /usr
8.4G    /var
14G     /home

---

Step 3: Check Large Files and Inode-Heavy Directories

03

Inspect Large Files and Many Small Files

Narrow the Cause

Search within the affected filesystem for unusually large regular files. If block space is available but inode use is high, count entries below likely small-file paths and identify parent directories with many regular files. Directories consume inodes too. Review each result with its service owner before cleanup.

# Example: list files larger than 500 MiB on the root filesystem.
sudo find / -xdev -type f -size +500M -printf '%s %p\n' 2>/dev/null |
  sort -n | tail -n 20

# Count all filesystem entries below a suspected cache directory; this
# includes directories as well as files, both of which consume inodes.
sudo find /var/cache -xdev -mindepth 1 -printf . | wc -c

# Find parent directories containing the most regular files.
sudo find /var/cache -xdev -type f -printf '%h\n' |
  sort | uniq -c | sort -nr | head -n 10

❯ View Expected Console Output

1610612736 /var/log/example-service/archive.log
45231
 12458 /var/cache/package-index

---

Step 4: Look for Deleted Files Still Held Open

04

Compare df and du Results

Open File Handles

If df reports much more use than du can account for, a process may still hold a deleted file open. If lsof is installed, +L1 lists open files with fewer than one link. Identify the owning process and service; space is normally released when the process closes the file, often after a controlled service restart.

sudo lsof -nP +L1

❯ View Expected Console Output

COMMAND   PID USER FD  TYPE DEVICE SIZE/OFF NLINK NAME
service  2345 root  7w  REG  252,1  2.0G    0     /var/log/service.log (deleted)

Figure 1: Disk checks reveal a nearly full filesystem and a large deleted log file still held open by rsyslogd.

---

Step 5: Reclaim Space Through the Owning Service

05

Use the Supported Cleanup or Rotation Path

Recovery

Prefer the package manager’s cache cleanup, the application’s retention controls, or the system’s log rotation policy. For systemd journal files, inspect the current size and apply a limit only when it matches your retention requirements. Recheck both blocks and inodes after cleanup to confirm the intended filesystem recovered capacity.

# Package cache examples; use the command for your distribution.
sudo apt clean
# or
sudo dnf clean all

# Inspect journal usage before considering any journal cleanup.
journalctl --disk-usage
df -hT
df -ih

❯ View Expected Console Output

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/vda1      ext4   40G   35G  4.2G  90% /

Do Not Remove Active Data to Make the Alert Disappear

Never use a broad rm command against logs, database files, or application data as a first response. Removing a file that a process still has open may not free space, and deleting active data can cause data loss. Identify the owner and use its documented cleanup or rotation procedure.

See the df manual and lsof manual for command behavior and options.