Troubleshoot High Load Average in Linux

uptime prints load average: 8.42, 6.11, 4.03 while top shows ninety percent idle — that combination confuses teams who treat load as “CPU percent.” On Linux, load counts how many tasks want the CPU or sit in uninterruptible sleep, averaged over one, five, and fifteen minutes.

The workflow below compares load to your CPU count, splits user, system, and I/O wait time, hunts D state blockers, and ties the signals together so you know whether to fix CPU, disk, or something waiting on the kernel.

What Linux load average actually measures

Load average is not CPU utilization. It is a moving average of runnable and uninterruptible tasks — processes ready to run plus processes stuck in D state waiting on kernel I/O.

Read the three numbers from uptime or /proc/loadavg:

uptime

09:22:27 up  1:01,  5 users,  load average: 0.25, 0.17, 0.27

The values are the one-, five-, and fifteen-minute averages. A single spike in the first number with lower long-term values often means a brief burst already fading. All three numbers climbing together means sustained pressure.

The same figures appear in top and in /proc/loadavg when scripts poll the host:

cat /proc/loadavg

0.25 0.17 0.27 1/754 21271

The fourth field (1/754 here) is runnable tasks over total tasks — useful when you graph load alongside process counts.

Compare load with available CPUs

Load only makes sense next to how many CPUs the kernel can schedule. Count logical processors:

nproc

3

On this lab VM, load averages around 0.25 mean far fewer than three tasks are competing for CPU time — a healthy idle host. Rule of thumb: sustained load above nproc means the run queue or uninterruptible sleep backlog is larger than the CPU can drain quickly.

Cross-check with lscpu when you need thread and socket detail for sizing:

lscpu | grep -E '^CPU\(s\)|^Model name'

CPU(s):                                  3
Model name:                              Intel(R) Core(TM) Ultra 5 135U

Hyper-threading and guest vCPUs both count toward nproc. A 16-thread laptop can tolerate higher load numbers than a 2-vCPU VM before users feel lag.

Check CPU utilization

Once you know load and CPU count, split where time goes — user, system, idle, and I/O wait. mpstat from the sysstat package prints per-interval CPU breakdown:

mpstat -P ALL 1 2

On an idle host, %idle stays high and %iowait stays near zero:

Average:     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
Average:     all    2.73    0.00    3.07    0.00    3.24    0.51    0.00    0.00    0.00   90.44

On the lab host I started CPU stress in one session:

stress --cpu 3 --timeout 8

While stress runs, sample again with mpstat in another terminal — user and system time climb while idle drops:

mpstat -P ALL 1 2

Average:     all   50.00    0.00   29.41    0.00   14.71    2.94    0.00    0.00    0.00    2.94

High %usr with load near nproc usually means CPU-bound work — find the top consumers:

pidstat -u 1 2

Sort by CPU in top when you need a live view — the first summary line repeats load and CPU states:

top -b -n 1 | head -5

top - 09:22:38 up  1:01,  7 users,  load average: 0.36, 0.19, 0.28
Tasks: 282 total,   1 running, 281 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.0 us,  0.0 sy,  0.0 ni, 91.4 id,  0.0 wa,  5.7 hi,  2.9 si,  0.0 st

The %Cpu(s) line labels match mpstat: us user, sy system, id idle, wa I/O wait. When load is high but us and sy stay low, look at I/O wait and D state next — not more CPU cores.

Check I/O wait

%iowait is the share of CPU time spent idle while at least one task waits on disk I/O. It explains “slow server” reports when load is elevated but user CPU looks fine.

Poll with mpstat and watch the %iowait column:

mpstat 1 3

Average:     all    1.27    0.00    4.98    1.16    6.94    1.04    0.00    0.00    0.00   84.61

Sustained %iowait above roughly ten to twenty percent on a busy server — context matters — often means disks or the storage path are the bottleneck. Pair it with iostat in the disk section below.

vmstat puts runnable and blocked counts beside I/O wait in one row:

vmstat 1 3

procs -----------memory---------- ---swap-- -----io---- -system-- -------cpu-------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st gu
 1  0      0 2103876   5052 3396940    0    0   844   249  900    4  2  7 91  0  0  0

r is runnable tasks, b is blocked in uninterruptible sleep, and wa is I/O wait percentage — the same story as mpstat in a compact snapshot.

Find processes in D state

Processes in D (uninterruptible sleep) wait on kernel I/O and count toward load even when they use no CPU time. List them:

ps -eo stat,pid,comm | awk '$1 ~ /D/'

No output on a healthy idle lab host means nothing is stuck in D right now — that is the result you want during normal operation.

When output appears, the STAT column shows D and the command name points at the blocked workload — often dd, database writers, or NFS clients. Those processes cannot be killed with SIGKILL until the kernel completes the I/O or times out the mount.

Count blocked tasks quickly:

ps -eo state | grep -c '^D'

0

A non-zero count with rising load and low %usr is a strong signal to inspect disks or NFS, not to add CPU.

Diagnose high load with low CPU

“High load, low CPU” is a common search phrase because load and CPU utilization measure different things. Load includes:

• Runnable tasks waiting for CPU time
• Tasks in D state blocked on I/O
• Short bursts that average into the one-minute figure

When top shows high idle and load still exceeds nproc, check three places in order:

1. vmstat — rising b (blocked) with moderate r
2. mpstat — rising %iowait even when %usr is low
3. ps — processes in D state

On the lab host, stress --io 4 raised the one-minute load without maxing user CPU:

uptime

09:23:37 up  1:02,  5 users,  load average: 0.95, 0.36, 0.33

0.95 on three CPUs is a modest queue — on production hardware, the same pattern at 12 on four CPUs with %iowait at thirty percent points at storage, not a need for faster application code. Confirm memory pressure separately if vmstat shows swap in (si/so); see check memory usage per process when free and swap traffic look wrong.

Check disk, NFS, and other blocking I/O

When %iowait or D state implicates I/O, identify which device is saturated. iostat reports per-disk utilization and queue depth:

iostat -xz 1 2

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           2.27    0.00    6.57    0.30    0.00   90.85

Device            r/s     w/s     rkB/s     wkB/s   w_await  aqu-sz  %util
sda             33.81    6.33    818.56    139.70     6.46    0.09   2.13

Focus on %util near one hundred percent and w_await / r_await in milliseconds — disks that stay fully utilized with high await times back up the whole system and inflate load.

Per-process disk reads and writes use pidstat:

pidstat -d 1 2

When the heavy consumers are NFS clients, check mounted NFS paths and server health:

mount | grep nfs

Stale or hung NFS mounts often leave processes in D until the server responds — soft versus hard mount options change whether tasks fail or wait indefinitely. For deeper disk tuning, see monitor disk I/O performance and how to improve disk I/O performance.

Identify the actual bottleneck

Match the dominant signal to the fix — chasing CPU when disks are saturated wastes effort.

Re-check uptime after each change — the one-minute average should fall within a few minutes if you removed the real blocker:

uptime

09:23:22 up  1:02,  6 users,  load average: 0.55, 0.26, 0.30

Load dropped after stress exited, which is the pattern you want after killing a runaway job or fixing a stuck mount.

Troubleshooting

References

• proc(5) — Linux manual page (/proc/loadavg)
• mpstat(1) — Linux manual page
• iostat(1) — Linux manual page
• vmstat(8) — Linux manual page
• ps(1) — Linux manual page

Summary

High load average on Linux means runnable or uninterruptible tasks are backing up — not automatically that CPUs are at one hundred percent. Compare the three load figures from uptime to nproc first; sustained load above the CPU count deserves investigation.

mpstat and top split user, system, idle, and I/O wait time. High %usr with load near the CPU count is CPU-bound work; high load with low user CPU and rising %iowait or vmstat b points at disk or NFS blocking. ps for D state finds tasks stuck in the kernel that still inflate load.

iostat names saturated disks through %util and await times; pidstat ties I/O to processes. Use the bottleneck table to pick whether you need more CPU, faster storage, a fixed mount, or memory — not all four at once. For recurring disk pressure, continue with monitor disk I/O performance; for runaway processes, see Linux process management.