Troubleshoot RHEL 9 Multipath and SAN Path Failures

Admin

October 11, 2026

13 min read

Device Mapper Multipath presents several SCSI paths to one storage LUN as a single block device. A degraded path can reduce redundancy; losing all paths can leave application I/O waiting or returning errors according to the configured policy. This guide investigates an existing RHEL 9 SCSI DM Multipath deployment with the storage and SAN teams.

Map the Application Volume to Its LUN WWID

Identity

Start with the affected mount and follow its filesystem, LV, physical volume, and multipath map. Record the persistent WWID and the array volume name. User-friendly aliases can differ between hosts; an alias such as mpatha is not enough to identify shared storage. Capture the current mapping before rescans or changes, since device names can move across reboots.

findmnt --target /srv/database
lsblk -e 7 -o NAME,KNAME,TYPE,SIZE,FSTYPE,MOUNTPOINTS,WWN
sudo pvs -o pv_name,vg_name,pv_uuid
sudo lvs -o lv_name,vg_name,lv_size,devices
sudo multipath -ll

❯ What to Verify

Application -> filesystem/LV -> multipath map -> LUN WWID
                                         |-> HBA/fabric path A -> array
                                         |-> HBA/fabric path B -> array

The incident's mount is tied to one verified WWID and its expected path count.

Inspect the Daemon, Map, and Path States

Topology

Check that multipathd is running, then compare the map topology with the expected number of paths and controllers. Read the device-mapper path state, checker result, and underlying SCSI device state together. A failed checker can coexist with a SCSI device marked running. In ALUA deployments, an enabled standby path group can be intentional; do not require every group to be active.

rpm -q device-mapper-multipath
systemctl status multipathd --no-pager
sudo multipath -ll
sudo multipathd show maps
sudo multipathd show paths

Correlate Path Changes with Host and Fabric Events

Timeline

Read multipathd and kernel events around the first slow or failed application request. Note path removals, SCSI command timeouts, transport resets, controller transitions, and filesystem errors. Ask the SAN team to correlate the HBA ports, switch ports, zoning, LUN masking, and array events using the same timestamps. A map still present in /dev/mapper does not prove it has usable paths.

sudo journalctl -u multipathd --since '30 minutes ago' --no-pager
sudo journalctl -k --since '30 minutes ago' --no-pager
# If sysstat is already installed:
iostat -xz 1 5
# For Fibre Channel deployments:
for host in /sys/class/fc_host/host*; do
  test -d "$host" || continue
  printf '\n%s\n' "$host"
  cat "$host/port_name" "$host/port_state"
done

Read the Effective Failover and Queueing Policy

Configuration

Inspect the effective configuration, including vendor/device overrides and per-WWID settings. Pay attention to no_path_retry and any queue_if_no_path behavior. When every path is unavailable, indefinite queueing can leave processes waiting for storage rather than returning an immediate error. The right policy depends on the application’s timeout and recovery design. Do not replace the vendor-supported profile with a generic failover example during the incident.

sudo multipath -t
sudo multipathd show config
sudo cat /etc/multipath.conf
ps -eo pid,stat,wchan:32,comm | awk '$2 ~ /^D/ {print}'

Restore the Failed Path with the Storage Owner

Targeted Repair

Repair the confirmed cable, switch, HBA, session, masking, or array issue. The daemon normally notices restored existing paths through its checks; monitor that transition before changing host configuration. If the storage presentation changed, use the array/vendor’s targeted rescan procedure for the identified hosts and LUNs. Avoid scanning, deleting, or reinitializing every device as a broad experiment.

# Observe recovery after the external path repair.
sudo multipathd show paths
sudo multipath -ll
sudo journalctl -u multipathd --since '10 minutes ago' --no-pager
# Compare the restored paths' WWID with Step 1 before accepting the result.

Reload Only a Reviewed Configuration Change

Optional Change

If the investigation establishes a configuration fault, preserve the existing file, apply the approved vendor-specific change, and inspect the parsed configuration before asking multipathd to reconfigure. This is a live change: grouping, checking, and queue policies can affect I/O. Reconfiguration can also restore queueing that was disabled at runtime. A fabric-only repair does not require an unrelated configuration edit.

# Approved change window only; stop if this incident backup name already exists.
sudo test -f /etc/multipath.conf || { echo 'Expected multipath.conf is missing; stop.' >&2; exit 1; }
sudo test ! -e /etc/multipath.conf.incident01-before || { echo 'Backup path already exists; choose a new incident-specific name.' >&2; exit 1; }
sudo test ! -L /etc/multipath.conf.incident01-before || { echo 'Backup path is a symlink; inspect it before continuing.' >&2; exit 1; }
sudo cp -a /etc/multipath.conf /etc/multipath.conf.incident01-before || exit 1
sudoedit /etc/multipath.conf
sudo multipath -t
# Continue only after the parsed configuration is accepted.
sudo multipathd reconfigure
sudo multipathd show config
sudo multipath -ll

Release Queueing for One Map Only When Required

Emergency Option

Confirm the map alias against its WWID immediately before the command. Disable queueing only for that map when the incident plan requires an error instead of indefinite waiting. Observe application and filesystem responses. Restore the configured queueing behavior after the storage path is recovered and the application owner approves it. Restoring the setting cannot undo transactions that already failed.

# Emergency, approved example only. Replace mpatha with the verified map.
sudo multipathd disablequeueing map mpatha
# After path recovery and the approved application/storage decision:
sudo multipathd restorequeueing map mpatha
sudo multipath -ll

Verify Redundancy, Filesystem State, and Workload Recovery

Closeout

Confirm the expected path count and ALUA groups, mounted filesystem options, absence of new block errors, and actual application operations. Use approved application health transactions instead of raw destructive disk writes. Schedule a separate maintenance test for path failover; do not pull a production path during closeout. Document the failed fault domain and any temporary queue-policy change that still needs a permanent design decision.

sudo multipath -ll
findmnt --target /srv/database -o SOURCE,TARGET,FSTYPE,OPTIONS
sudo journalctl -k --since '10 minutes ago' --no-pager
iostat -xz 1 5
# Run the storage/application owner's approved health transaction.