Linux "top" command: What are us, sy, ni, id, wa, hi, si and st (for CPU usage)?

When I issue top in Linux, I get a result similar to this:

Screenshot of top

One of the lines has CPU usage information represented like this:

Cpu(s): 87.3%us,  1.2%sy,  0.0%ni, 27.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

While I know the definitions of each of them (far below), I don’t understand what these tasks exactly mean.

  • hi – what does servicing hardware interrupts mean?
  • si – what does servicing software interrupts mean?
  • st – they say it’s the “CPU time in involuntary wait by the virtual CPU while hypervisor is servicing another processor (or) % CPU time stolen from a virtual machine”.

But what does it actually mean? Can someone be more clear?

I listed all of us, sy, ni, etc, because it could help others searching for the same. This information is not in the man pages.

us: user cpu time (or) % CPU time spent in user space
sy: system cpu time (or) % CPU time spent in kernel space
ni: user nice cpu time (or) % CPU time spent on low priority processes
id: idle cpu time (or) % CPU time spent idle
wa: io wait cpu time (or) % CPU time spent in wait (on disk)
hi: hardware irq (or) % CPU time spent servicing/handling hardware interrupts
si: software irq (or) % CPU time spent servicing/handling software interrupts
st: steal time - - % CPU time in involuntary wait by virtual cpu while hypervisor is servicing another processor (or) % CPU time stolen from a virtual machine
Asked By: its_me

||

hi is the time spent processing hardware interrupts. Hardware interrupts are generated by hardware devices (network cards, keyboard controller, external timer, hardware sensors, …) when they need to signal something to the CPU (data has arrived, for example).

Since these can happen very frequently, and since they essentially block the current CPU while they are running, kernel hardware interrupt handlers are written to be as fast and simple as possible.

If long or complex processing needs to be done, these tasks are deferred using a mechanism call softirqs. These are scheduled independently, can run on any CPU, can even run concurrently (none of that is true of hardware interrupt handlers).

The part about hard IRQs blocking the current CPU, and the part about softirqs being able to run anywhere are not exactly correct, there can be limitations, and some hard IRQs can interrupt others.

As an example, a “data received” hardware interrupt from a network card could simply store the information “card ethX needs to be serviced” somewhere and schedule a softirq. The softirq would be the thing that triggers the actual packet routing.

si represents the time spent in these softirqs.

A good read about the softirq mechanism (with a bit of history too) is Matthew Wilcox’s I’ll Do It Later: Softirqs, Tasklets, Bottom Halves, Task Queues,
Work Queues and Timers
(PDF, 64k).

st, “steal time”, is only relevant in virtualized environments. It represents time when the real CPU was not available to the current virtual machine — it was “stolen” from that VM by the hypervisor (either to run another VM, or for its own needs).

The CPU time accounting document from IBM has more information about steal time, and CPU accounting in virtualized environments. (It’s aimed at zSeries type hardware, but the general idea is the same for most platforms.)

Answered By: Mat
  • us – Time spent in user space
  • sy – Time spent in kernel space
  • ni – Time spent running niced user processes (User defined priority)
  • id – Time spent in idle operations
  • wa – Time spent on waiting on IO peripherals (eg. disk)
  • hi – Time spent handling hardware interrupt routines. (Whenever a peripheral unit want attention form the CPU, it literally pulls a line, to signal the CPU to service it)
  • si – Time spent handling software interrupt routines. (a piece of code, calls an interrupt routine…)
  • st – Time spent on involuntary waits by virtual cpu while hypervisor is servicing another processor (stolen from a virtual machine)
Answered By: Simon Rigét

The “st” value can be simply explained by using an T2.micro EC2 instance from AWS.

In the AWS documentation you can read that you get only a 10% baseline performance per VCPU. This means that if you have a process which would consume a lot of cpu time, the “st” value will stay around 90 since you are only allowed to use 10% of the VCPU. The sum of the other values will stay around 10.

So AWS is using the hypervisor to only allow you access to a certain amount of computing power. It slows you down by intention since you are only using a low tier type of instance.

I hope this makes things a little bit easier to understand.

Answered By: draufunddran

Procs
r: The number of processes waiting for run time.
b: The number of processes in uninterruptible sleep.
Memory
swpd: the amount of virtual memory used.
free: the amount of idle memory.
buff: the amount of memory used as buffers.
cache: the amount of memory used as cache.
inact: the amount of inactive memory. (-a option)
active: the amount of active memory. (-a option)
Swap
si: Amount of memory swapped in from disk (/s).
so: Amount of memory swapped to disk (/s).
IO
bi: Blocks received from a block device (blocks/s).
bo: Blocks sent to a block device (blocks/s).
System
in: The number of interrupts per second, including the clock.
cs: The number of context switches per second.
CPU
These are percentages of total CPU time.
us: Time spent running non-kernel code. (user time, including nice time)
sy: Time spent running kernel code. (system time)
id: Time spent idle. Prior to Linux 2.5.41, this includes IO-wait time.
wa: Time spent waiting for IO. Prior to Linux 2.5.41, included in idle.
st: Time stolen from a virtual machine. Prior to Linux 2.6.11, unknown.

Answered By: João Paulo Fernandes
Categories: Answers Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.