| Tested on | RHEL 10.2 (Coughlan) — vm1.lab.example (3 logical CPUs) |
|---|---|
| Package | procps-ng 4.0.4-11.el10sysstat 12.7.6-4.el10util-linux 2.40.2-18.el10stress 1.0.7-5.el10_0 |
| Applies to | Ubuntu, Debian, Kali Linux, Linux Mint, Pop!_OS, Raspberry Pi OS, elementary OS, Zorin OS, Parrot OS, MX Linux, RHEL, Rocky Linux, AlmaLinux, Oracle Linux, CentOS Stream, Fedora, Arch Linux |
| Privilege | Normal user for top, ps, mpstat, and pidstat -u; sudo or root for pidstat on other users' processes and some strace options |
| Scope | Find and diagnose Linux high CPU usage with top, ps, mpstat, pidstat, per-core stats, threads, and /proc inspection. Does not cover application profilers, eBPF tracing, or cgroup CPU limits. |
| Related guides | High load average troubleshooting top command Linux process management Processors, cores, and threads stress command |
top shows %Cpu(s): 54.5 us, 18.2 sy, 0.0 id while users complain the box is crawling — that is a real Linux CPU troubleshooting scene. Before you add vCPUs or restart services, you need the process list, per-core view, and a split between user, system, and I/O wait time.
The steps below walk through that order on a three-CPU lab host: confirm overall CPU pressure, name the top consumers, check whether one core is pinned, then inspect the winning PID and decide if the burn is expected work or a runaway loop.
Check overall CPU usage
Start with a one-shot snapshot so you are not staring at a scrolling terminal. Batch mode top prints the summary line and the hottest processes once:
top -b -n 1 | head -14top - 09:26:57 up 1:05, 5 users, load average: 0.53, 0.40, 0.34
Tasks: 274 total, 4 running, 270 sleeping, 0 stopped, 0 zombie
%Cpu(s): 54.5 us, 18.2 sy, 0.0 ni, 0.0 id, 0.0 wa, 18.2 hi, 9.1 si, 0.0 st
MiB Mem : 7679.0 total, 1962.3 free, 2690.1 used, 3328.0 buff/cache
MiB Swap: 3076.0 total, 3076.0 free, 0.0 used. 4988.8 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22497 root 20 0 3152 564 448 R 10.1 0.0 0:03.52 stress
22498 root 20 0 3152 564 448 R 10.1 0.0 0:01.84 stress
22496 root 20 0 3152 564 448 R 9.5 0.0 0:03.46 stressThe %Cpu(s) row is the whole-machine view: us is user time, sy is kernel time, id is idle, and wa is I/O wait. Here 0.0 id means the CPUs were busy during the sample — exactly what you expect while stress --cpu 3 was running on this host.
uptime adds load averages beside CPU percent. Load and CPU answer different questions — see high load average troubleshooting when load is high but %idle stays high.
uptime09:26:29 up 1:05, 5 users, load average: 0.49, 0.37, 0.33On an idle host, load stays below the CPU count (3 here) and %idle in top stays high. When both load and user CPU climb together, you are usually looking at real compute work, not a stuck disk queue.
Find the highest CPU processes
ps with a sort flag is the fastest non-interactive way to list offenders. The %CPU column is an average since the process started, so pair it with top or pidstat for what is hot right now:
ps aux --sort=-%cpu | head -6USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 22497 98.0 0.0 3152 564 ? R 09:26 0:03 stress --cpu 3 --timeout 12
root 22496 96.1 0.0 3152 564 ? R 09:26 0:03 stress --cpu 3 --timeout 12
root 22498 51.2 0.0 3152 564 ? R 09:26 0:01 stress --cpu 3 --timeout 12Three stress workers sit at the top with R (running) state — that is the pattern you want when hunting a CPU spike. Ignore kernel threads unless %sys in top is unusually high; focus on user-owned application PIDs first.
For a live ranking every second, pidstat from sysstat names the process and splits user versus system time:
pidstat -u 1 2Average: UID PID %usr %system %guest %wait %CPU CPU Command
Average: 0 3376 0.50 0.99 0.00 0.50 1.49 - node
Average: 0 22513 0.00 0.50 0.00 0.00 0.50 - pidstatDuring the idle sample above, nothing dominated CPU. Re-run pidstat -u 1 5 while users report slowness — the %CPU column updates every second so you catch short cron bursts that ps averages away.
Check individual CPU cores
Overall %Cpu(s) hides a single core pinned at one hundred percent while the rest idle. mpstat -P ALL prints each logical CPU separately:
mpstat -P ALL 1 2Average: CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
Average: all 42.44 0.00 27.31 0.00 14.71 0.42 0.00 0.00 0.00 15.13
Average: 0 57.50 0.00 11.25 0.00 15.00 0.00 0.00 0.00 0.00 16.25
Average: 1 60.00 0.00 10.00 0.00 15.00 0.00 0.00 0.00 0.00 15.00
Average: 2 8.97 0.00 61.54 0.00 14.10 1.28 0.00 0.00 0.00 14.10All three CPUs show low %idle during the stress --cpu 3 run — the workload spread across cores. If only CPU 2 were saturated, you would suspect a single-threaded app or a process bound to one CPU with taskset.
Count logical processors when you interpret the numbers:
nproc3A host with nproc of 8 can run eight CPU-bound threads at full speed without queueing. One thread at one hundred percent on a single core still leaves seven cores mostly idle but can still make that application feel stuck.
Check user, system, and I/O wait CPU
The %Cpu(s) line in top and the all row in mpstat answer the same question in different layouts:
%usr— time in application code%sys— kernel time (syscalls, scheduling, drivers)%iowait— CPUs idle while waiting for disk I/O to finish%idle— truly unused CPU cycles
On an idle lab host, %idle stays near ninety-eight percent:
mpstat 1 2Average: CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice %idle
Average: all 0.17 0.00 0.17 0.00 1.51 0.17 0.00 0.00 0.00 97.99High %usr with low %iowait means CPU-bound work — optimize code, scale out, or add cores. High %iowait with moderate %usr means the bottleneck is storage; fixing CPU will not help until disk or NFS latency improves. Very high %sys with modest %usr points at kernel overhead — check pidstat -w for context switches, audit security agents, or recent driver changes.
Find high-CPU threads
A process can look modest in ps while one thread inside it burns a full core. top -H switches to thread view; pass -p to focus on one PID:
Start a short CPU worker so thread view has something to show:
stress --cpu 1 --timeout 8 &top -H lists threads; -p limits the table to the stress parent:
top -H -b -n 1 -p $(pgrep -n stress) | head -12Threads: 1 total, 1 running, 0 sleeping, 0 stopped, 0 zombie
%Cpu(s): 21.4 us, 0.0 sy, 0.0 ni, 67.9 id, 0.0 wa, 7.1 hi, 3.6 si, 0.0 st
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22408 root 20 0 3152 548 440 R 92.3 0.0 0:01.43 stressHere the thread PID (22408) matches the process because stress is single-threaded. In Java, database, or web server processes, one row near one hundred percent %CPU often identifies the hot worker thread while siblings sleep.
List threads with ps when you already know the parent PID:
ps -Lp $(pgrep -n stress) -o pid,tid,pcpu,commPID TID %CPU COMMAND
22518 22518 93.0 stressTID is the thread ID — the same number top -H shows in the PID column for threads. Pass that TID to strace -p TID only on non-production systems or with change approval; strace itself adds CPU overhead.
Inspect the problem process
Once you have a PID, confirm what is actually running before kill. ps shows owner, state, and elapsed time:
ps -p $(pgrep -n stress) -o pid,user,%cpu,%mem,stat,etime,cmdPID USER %CPU %MEM STAT ELAPSED CMD
22518 root 95.1 0.0 R 00:01 stress --cpu 1 --timeout 8R means running; S means sleeping. A process stuck at high %CPU in R for hours is a stronger kill candidate than a five-minute gcc build.
The executable path and full command line come from /proc:
readlink -f /proc/$(pgrep -n stress)/exe/usr/bin/stressThe full argument list — including flags your ps output may truncate — is in cmdline:
tr '\0' ' ' < /proc/$(pgrep -n stress)/cmdline; echostress --cpu 1 --timeout 8/proc/PID/status adds thread count and context-switch counters — useful when you wonder if the process is fighting for CPU time:
grep -E '^(Name|State|Threads|voluntary|nonvoluntary)' /proc/$(pgrep -n stress)/statusName: stress
State: R (running)
Threads: 1
voluntary_ctxt_switches: 1
nonvoluntary_ctxt_switches: 57Rising nonvoluntary_ctxt_switches under load can mean CPU contention or too many runnable threads for the core count. For service-managed apps, check systemctl status unit and recent deploys before you terminate the PID.
Determine why CPU usage is high
Match what you measured to the likely cause — not every spike needs more hardware.
| What you see | Likely cause | What to do |
|---|---|---|
Few PIDs at top of ps, high %usr, low %iowait |
Legitimate CPU work (compile, backup, batch) | Confirm schedule; renice or run off-peak; add cores if sustained |
Same command, many instances, climbing %CPU |
Runaway loop, fork bomb, or stuck cron overlap | Stop the service; fix script; add locking to cron |
One thread at ~100% on one core in top -H |
Single-threaded hot path | Profile or patch app; scale horizontally |
High %sys, moderate %usr |
Kernel, driver, or AV overhead | pidstat -w; review dmesg; update or tune agents |
High %iowait, low %usr |
Disk or NFS wait (not CPU shortage) | iostat -xz; see monitor disk I/O performance |
| CPU high after deploy | New code path or config | Roll back; compare ps command lines to previous build |
| CPU capped but process wants more | cgroup or container limit | Check cgroup CPU limits and orchestrator quotas |
After you stop or fix the hot process, confirm CPU returned to normal:
top -b -n 1 | head -5top - 09:26:31 up 1:05, 5 users, load average: 0.61, 0.40, 0.34
Tasks: 270 total, 1 running, 269 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 5.7 sy, 0.0 ni, 88.6 id, 0.0 wa, 5.7 hi, 0.0 si, 0.0 st88.6 id after stress exited is the recovery pattern you want. If %idle stays low but ps looks quiet, re-check per-core mpstat and thread view — the burner may be a kernel thread or a short-lived process you missed between samples.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
%CPU at 100% but only one core busy |
Single-threaded process | top -H; optimize thread pool or add instances |
Many stress-like unknown PIDs |
Compromised host or bad cron | Isolate; audit /etc/cron.* and user crontabs |
High CPU, %wa also high |
Storage bottleneck masquerading as CPU pain | iostat; fix disk — do not only add vCPUs |
java or node high CPU, unclear thread |
In-process hot loop | top -H; use app profiler in lower environment |
CPU normal in top, users still slow |
Network, DNS, or memory — not CPU | Check latency and swap; see check memory usage per process |
| Process returns immediately after kill | systemd or supervisor restart | Fix unit; disable restart loop; patch root cause |
%steal high on VM |
Hypervisor oversubscription | Move VM or request reserved CPU on cloud host |
References
- top(1) — Linux manual page
- ps(1) — Linux manual page
- mpstat(1) — Linux manual page
- pidstat(1) — Linux manual page
- proc(5) — Linux manual page
Summary
High CPU usage on Linux starts with a snapshot: top -b or mpstat tells you whether the machine is genuinely compute-bound or mostly waiting on I/O. ps aux --sort=-%cpu and pidstat -u name the processes responsible; run them during the incident, not five minutes later, because %CPU in ps is a lifetime average.
Split user, system, and I/O wait before you buy hardware. High %usr on a few PIDs is application work; high %iowait is a storage problem; high %sys deserves a kernel and agent review. Use mpstat -P ALL and top -H when one core is maxed but the summary line still shows idle headroom — single-threaded loops and hot worker threads hide there.
Inspect /proc/PID/cmdline and ps before you kill anything. Legitimate batch jobs spike CPU safely; runaway scripts and fork loops need a service stop or code fix. When limits rather than load are the story, check cgroup and container CPU caps next.
