The iostat command is a system monitoring tool in Linux that provides comprehensive usage information on input/output (I/O) devices, partitions, and the CPU. This information can be used in analyzing resource allocation and utilization, identifying potential performance issues, and troubleshooting.
iostat monitors the active time of I/O devices in relation to their average transfer rates. It generates these reports mainly by gathering information from the /proc filesystem, including data related to the system uptime, disk metrics, and system metrics.
Hence, the /proc filesystem must be mounted for iostat to function optimally.
In this article, we'll cover some handy iostat commands, which you can use to gain valuable insights into your system's performance.
Types of Reports
Generally, iostat produces two types of reports:
- CPU Utilization Report: This contains information on the overall CPU performance of the system. Metrics tracked include idle CPU time, CPU utilization at the application and kernel level, and CPU wait time. Furthermore, in systems with multiprocessors, the values produced are the total averages among all processors.
- Device Utilization Report: This shows information on the utilization of block devices and partitions. Key metrics tracked include read and write requests, wait times for I/O requests, and data transfer rates.
Install iostat
To begin using iostat, the sysstat package, a collection of system monitoring tools, which includes iostat must be installed.
For Ubuntu and other Debian-based distributions, run:
sudo apt install sysstat
On CentOs, Fedora, and RHEL, run:
sudo dnf install sysstat
On Arch-based distributions, run:
sudo pacman -S sysstat
iostat Command Syntax
Generally, iostat commands have the following syntax:
iostat [options] [devices] [interval] [count]
- options: These are flags that customize the behavior of the iostat command. Commonly used options include -c, -h, and -d.
- devices: This defines the block devices for which performance metrics will be monitored.
- interval: This indicates the duration (in seconds) between successive reports.
- count: This indicates the number of times reports are to be generated. If omitted and the interval is specified, reports are continuously generated.
1. Display All Reports
Issuing the iostat command without additional flags will produce a report showing both CPU and device utilization reports. Furthermore, when invoking iostat for the first time, the report produced usually shows information from when the system was booted. However, subsequent reports cover the period since the last report was made.
iostat
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 01/10/2023 _x86_64_(2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
6.83 0.43 8.24 0.22 0.00 84.27
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
loop0 0.04 0.04 0.00 0.00 17 0 0
loop1 0.12 0.90 0.00 0.00 350 0 0
loop2 0.64 7.69 0.00 0.00 2986 0 0
loop3 0.13 0.93 0.00 0.00 360 0 0
loop4 0.14 2.75 0.00 0.00 1068 0 0
sda 47.78 1612.70 106.36 0.00 626452 41317 0
sdb 0.62 14.48 0.27 0.00 5624 104 0
sr0 0.16 5.39 0.00 0.00 2092 0 0
sr1 0.16 5.40 0.00 0.00 2096 0 0
The output produced above shows two tables — the first table represents the CPU utilization report, while the second table is the device utilization report for all block devices and partitions the system uses.
The CPU utilization report highlights the following information:
- %user: This column indicates the percentage of CPU time spent executing processes at the user level.
- %nice: This shows the percentage of CPU time spent executing processes with a nice priority.
- %system: This represents the percentage of CPU time spent executing system-level (kernel) tasks.
- %iowait: This shows the percentage of CPU time during which the CPU was idle, waiting for the completion of I/O operations. Furthermore, high values could indicate that the CPU was expecting data from storage devices.
- %steal: In a virtualized environment, this represents the percentage of CPU time spent waiting while the underlying hypervisor serviced another virtual machine.
- %idle: This shows the percentage of time for which the CPU was idle, without executing any active processes. Moreover, the 84% value in the example above indicates that the CPU has sufficient processing capacity.
Similarly, the device utilization report highlights the following information:
- Device: This column indicates the names of block devices or partitions.
- tps: This shows the number of transfers sent to the device per second. In other words, it represents the I/O requests sent to the device.
- kB_read/s: Also, MB_read/s or Blk_read/s depending on the unit used. It indicates the amount of data read from a device, expressed in kilobytes, megabytes, or blocks per second.
- kB_wrtn/s: Also, MB_wrtn/s or Blk_wrtn/s. It shows the amount of data written to a device, expressed in kilobytes, megabytes, or blocks per second.
- kB_dscd/s: Also, MB_dscd/s or Blk_dscd/s. This represents the amount of discarded data for a block device, expressed in kilobytes, megabytes, or blocks per second.
- kB_read: Also, MB_read or Blk_read. This indicates the total amount of data retrieved from a block device since boot, expressed in kilobytes, megabytes, or blocks.
- kB_wrtn: Also, MB_wrtn or Blk_wrtn. This shows the total amount of data written to a block device since boot, expressed in kilobytes, megabytes, or blocks.
- kB_dscd: Also MB_dscd or Blk_dscd. This shows the total amount of data discarded for a block device since boot, expressed in kilobytes, megabytes, or blocks.
Additionally, the "-p" flag can be added to the command, which instructs iostat to include partitions in the generated report.
iostat -p
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 01/10/2023 _x86_64_(2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
6.83 0.43 8.24 0.22 0.00 84.27
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
loop0 0.04 0.04 0.00 0.00 17 0 0
loop1 0.12 0.90 0.00 0.00 350 0 0
loop2 0.64 7.69 0.00 0.00 2986 0 0
loop3 0.13 0.93 0.00 0.00 360 0 0
loop4 0.14 2.75 0.00 0.00 1068 0 0
sda 9.79 109.64 42.26 0.00 629080 242477 0
sda1 0.00 0.02 0.00 0.00 104 0 0
sda2 0.03 0.99 0.00 0.00 5688 1 0
sda3 9.74 108.25 42.26 0.00 621100 242476 0
sdb 0.04 0.98 0.02 0.00 5624 104 0
sdb1 0.00 0.18 0.00 0.00 1040 0 0
sr0 0.16 5.39 0.00 0.00 2092 0 0
sr1 0.16 5.40 0.00 0.00 2096 0 0
From the output above, the partitions — sda1, sda1, sda3, and sdb1 are now included in the report.
2. Show Only CPU Utilization Report
Generally, invoking the iostat command produces both types of reports. However, you can instruct iostat to show only the CPU utilization report by adding the "-c" flag.
iostat -c
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 01/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
2.00 0.03 2.29 0.05 0.00 95.63
3. Show Only Device Utilization Report
Similarly, you can instruct iostat to show only the device utilization report by adding the "-d" flag instead.
iostat -d Linux 6.2.0-33-generic (ubuntu-virtual-machine) 01/10/2023 _x86_64_ (2 CPU) Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd loop0 0.04 0.04 0.00 0.00 17 0 0 loop1 0.12 0.90 0.00 0.00 350 0 0 loop2 0.64 7.69 0.00 0.00 2986 0 0 loop3 0.13 0.93 0.00 0.00 360 0 0 loop4 0.14 2.75 0.00 0.00 1068 0 0 sda 47.78 1612.70 106.36 0.00 626452 41317 0 sdb 0.62 14.48 0.27 0.00 5624 104 0 sr0 0.16 5.39 0.00 0.00 2092 0 0 sr1 0.16 5.40 0.00 0.00 2096 0 0
4. Show Report For Selected Devices
Sometimes the produced report may be too tedious to go through, especially with the inclusion of several loop devices. However, iostat allows you to generate reports for specific block devices by simply specifying their names.
For example, the following command produces a report for both the /dev/sda and /dev/sdb block devices.
iostat sda sdb
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 01/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.96 0.03 2.24 0.05 0.00 95.72
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 9.65 102.59 42.08 0.00 629080 258025 0
sdb 0.02 0.39 0.01 0.00 5624 104 0
Similarly, reports for a single block device can be produced as follows:
iostat sda
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 01/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.96 0.03 2.24 0.05 0.00 95.72
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 9.65 102.59 42.08 0.00 629080 258025 0
Additionally, when you add the "-p" flag to the command, the partitions of the block device are included in the report as demonstrated below.
iostat -p sda
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 01/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.96 0.0 2.24 0.05 0.00 95.72
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 9.65 102.59 42.08 0.00 629080 258025 0
sda1 0.00 0.02 0.00 0.00 104 0 0
sda2 0.02 0.84 0.00 0.00 5688 1 0
sda3 9.41 91.22 41.90 0.00 621100 285272 0
5. Show Metrics Summary
Assuming you want to retrieve a summary of the metrics generated for all devices or selected devices, iostat allows you to do this easily with the help of the "-g" flag. This flag takes a name argument, which represents the row for the aggregate data produced.
For instance, the command below obtains a summary of I/O metrics for devices, /dev/sda and /dev/sdb.
iostat - g Total sda sdb
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 02/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.95 0.07 2.22 0.05 0.00 95.71
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 8.91 56.95 77.09 0.00 853628 1155449 0
sdb 0.02 0.38 0.01 0.00 5624 104 0
Total 8.92 57.33 77.10 0.00 859252 1155553 0
In the output above, a new row, Total, which represents the aggregate data for each field is included.
Additionally, the "-H" flag can be combined with the "-g" flag, which instructs iostat to show only the summary row in the output.
iostat -g Total sda sdb -H
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 02/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.95 0.07 2.22 0.05 0.00 95.71
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
Total 8.92 57.33 77.10 0.00 859252 1155553 0
6. Show Report at N Intervals
So far, you've only worked with a single report history. However, iostat can also produce reports at regular intervals, which allows you to gain real-time insights into the system's performance.
To generate reports after certain intervals, a numeric value, representing the period in seconds between successive reports is added to the command.
For example, the command below produces a device utilization report for the /dev/sda block device every 3 seconds.
iostat -d sda 3 Linux 6.2.0-33-generic (ubuntu-virtual-machine) 02/10/2023 _x86_64_ (2 CPU) Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd sda 8.84 49.17 72.34 0.00 853628 1255801 0 Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd sda 1.00 0.00 12.00 0.00 0 36 0 Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd sda 0.33 0.00 1.33 0.00 0 4 0
Consequently, a report is continuously generated until an interrupt signal is sent.
7. Show Report N Times
If you want to generate a limited number of reports, iostat provides an easy way to achieve this. Basically, the second numeric value passed to the iostat command represents the number of times reports will be produced.
For example, the following command produces device utilization reports for the /dev/sda block device 2 times with a 5-second interval.
iostat -d sda 5 2 iostat -d sda 3 Linux 6.2.0-33-generic (ubuntu-virtual-machine) 02/10/2023 _x86_64_ (2 CPU) Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd sda 8.84 49.17 72.34 0.00 853628 1255801 0 Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd sda 1.00 0.00 12.00 0.00 0 36 0
8. Remove Inactive Device Metrics
By default, when iostat generates reports at intervals, the report produced also includes block devices and partitions that may have been inactive during that period. However, iostat allows you to limit generated reports to only active devices by adding the "-z" flag.
For instance, the command below produces a report 3 times in 5-second intervals and filters out any inactive devices.
iostat -z 5 3
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 03/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.89 0.05 2.15 0.05 0.00 95.86
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
loop0 0.00 0.00 0.00 0.00 17 0 0
loop1 0.00 0.02 0.00 0.00 350 0 0
loop2 0.01 0.16 0.00 0.00 2986 0 0
loop3 0.00 0.02 0.00 0.00 360 0 0
loop4 0.00 0.06 0.00 0.00 1068 0 0
loop5 0.00 0.06 0.00 0.00 1113 0 0
sda 8.81 44.47 69.56 0.00 855852 1338873 0
sdb 0.01 0.29 0.01 0.00 5624 104 0
sr0 0.00 0.11 0.00 0.00 2092 0 0
sr1 0.00 0.11 0.00 0.00 2096 0 0
avg-cpu: %user %nice %system %iowait %steal %idle
2.23 0.00 2.84 0.00 0.00 94.92
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 44.40 0.00 214.40 0.00 0 1072 0
avg-cpu: %user %nice %system %iowait %steal %idle
1.82 0.00 1.72 0.00 0.00 96.46
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 1.20 0.00 9.60 0.00 0 48 0
In the output above, the first report includes all the devices the system uses. However, subsequent reports contain only the /dev/sda block device, which was the only active device during that timespan.
9. Show Extended Metrics
The device utilization reports produced so far have shown the abridged version of the table with only 7 metrics. However, the device utilization report is much more extensive than that, containing some vital metrics.
The "-x" flag is used to instruct iostat to show a comprehensive version of the report. Since this information can be a lot, the "--pretty" flag can be added to break the report into sub-reports as demonstrated below.
iostat sda -x --pretty
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 02/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
1.88 0.04 2.13 0.04 0.00 95.90
r/s rkB/s rrqm/s %rrqm r_await rareq-sz Device
0.73 37.58 0.28 27.40 0.88 51.18 sda
w/s wkB/s wrqm/s %wrqm w_await wareq-sz Device
8.07 65.31 1.30 13.90 1.22 8.10 sda
d/s dkB/s drqm/s %drqm d_await dareq-sz Device
0.00 0.00 0.00 0.00 0.00 0.00 sda
f/s f_await aqu-sz %util Device
0.00 0.00 0.01 0.42 sda
Some important information highlighted in the report includes:
- r/s: This indicates the number of completed read requests per second for the device.
- %rrqm: This represents the percentage of read requests merged before being sent to the device. Moreover, high values indicate efficient I/O operations.
- r_wait: This shows the average time (in milliseconds) it takes for read requests sent to the device to be completed.
- w/s: This indicates the number of completed write requests per second for the device.
- %wrqm: This shows the percentage of write requests merged before being sent to the device.
- w_wait: This shows the average time (in milliseconds) it takes for write requests sent to the device to be completed.
- %drqm: This indicates the percentage of discard requests merged before being sent to the device.
- aqu-sz: This shows the average queue length of requests waiting to be handled by the device.
- %util: This indicates the percentage of time spent by the block device in handling I/O requests.
10. Change Metrics Unit
The device utilization report uses kilobytes as the default unit. However, iostat provides two flags "—k" and "-m" for modifying the unit depending on your preference. When the "-m" flag is used, the metrics are expressed in megabytes. Whereas, the "-k" flag sets the metrics in kilobytes.
For example, the following command changes the unit of the produced report to megabytes.
iostat sda -m
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 05/10/2023 _x86_64_(2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
23.42 0.85 32.80 1.13 0.00 41.81
Device tps MB_read/s MB_wrtn/s MB_dscd/s MB_read MB_wrtn MB_dscd
sda 195.88 7.42 0.30 0.00 522 21 0
11. Show Report in Human-readable Format
Adding the "-h" flag to the iostat command produces a more intuitive report. Specifically, the metrics produced are suffixed with units such as k (kilobytes), m (megabytes), and % for CPU-related metrics.
iostat -h
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 02/10/2023 _x86_64_ (2 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
4.9% 0.2% 7.0% 0.3% 0.0% 87.5%
tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd Device
0.01 0.0k 0.0k 0.0k 17.0k 0.0k 0.0k loop0
0.05 0.4k 0.0k 0.0k 350.0k 0.0k 0.0k loop1
0.05 0.4k 0.0k 0.0k 360.0k 0.0k 0.0k loop2
0.06 1.1k 0.0k 0.0k 1.0M 0.0k 0.0k loop3
0.25 3.0k 0.0k 0.0k 2.9M 0.0k 0.0k loop4
0.06 1.1k 0.0k 0.0k 1.1M 0.0k 0.0k loop5
36.63 862.9k 1.2M 0.0k 839.5M 1.1G 0.0k sda
0.17 4.3k 0.0k 0.0k 4.1M 0.0k 0.0k sdb
0.06 2.1k 0.0k 0.0k 2.0M 0.0k 0.0k sr0
0.08 2.1k 0.0k 0.0k 2.0M 0.0k 0.0k sr1
12. Show Report Timestamp
Adding the "-t" flag to the iostat command ensures that timestamps are included with each report. This allows you to easily track the system's performance over time and to have a timeline for the generated reports.
iostat -t sda 4 2
Linux 6.2.0-33-generic (ubuntu-virtual-machine) 05/10/2023 _x86_64_ (2 CPU)
05/10/2023 04:20:37
avg-cpu: %user %nice %system %iowait %steal %idle
4.17 0.12 5.69 0.30 0.00 89.72
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 31.79 609.16 975.27 0.00 861341 1379001 0
05/10/2023 04:20:41
avg-cpu: %user %nice %system %iowait %steal %idle
2.45 0.00 3.22 0.00 0.00 94.33
Device tps kB_read/s kB_wrtn/s kB_dscd/s kB_read kB_wrtn kB_dscd
sda 0.00 0.00 0.00 0.00 0 0 0
The output above includes the timestamps of the reports in human-readable format.
13. Show Reports in JSON
Javascript object notation (JSON) is a data interchange format consisting of key-value pairs, which allows data to be parsed easily. It is widely used for sending and receiving data.
iostat supports report generation in JSON, which is achieved by adding the "-o" flag together with the "JSON" argument to the command.
For example, the command below produces reports for the /dev/sda block device in JSON format.
iostat -o JSON sda
{"sysstat": {
"hosts": [
{
"nodename": "ubuntu-virtual-machine",
"sysname": "Linux",
"release": "6.2.0-33-generic",
"machine": "x86_64",
"number-of-cpus": 2,
"date": "05/10/2023",
"statistics": [
{
"avg-cpu": {"user": 2.72, "nice": 0.05, "system": 3.44, "iowait": 0.15, "steal": 0.00, "idle": 93.64},
"disk": [
{"disk_device": "sda", "tps": 17.04, "kB_read/s": 255.72, "kB_wrtn/s": 429.40, "kB_dscd/s": 0.00, "kB_read": 861453, "kB_wrtn": 1446541, "kB_dscd": 0}
]
}
]
}
]
}}
In the output above, the report produced is shown as a JSON array with the key statistics. The array contains two JSON objects, representing both the CPU utilization report and the device utilization report.
Conclusion
To sum up, the iostat command is a valuable tool for monitoring and analyzing both CPU and I/O device performance in Linux. It offers crucial insights into the utilization, throughput, and responsiveness of devices.
Reports produced by iostat allow you to make informed decisions on resource allocation and identify potential performance issues on your system.
