How to Test (Benchmark) Disk IO Speed with Sysbench in Linux / Ubuntu

By | May 11, 2023

Sysbench is a command line utility program available on linux that can be used to conduct different types of benchmark tests to measure performance of disk io, memory, cpu and mysql database.

In another article we learned how to use sysbench to test ram read/write speed.

In this particular article we take a quick look at how we can use the sysbench utility program to benchmark disk io speed.

For the sake of the this experiment we shall be used different disk drives like seagate external hdd, samsung external ssd, usb flash drives etc.

Install sysbench

The first thing that you need to do is to install sysbench. It is available in the repositories of most linux distros.

Ubuntu

sudo apt install sysbench

Setup the drive

The disk drive that you want to benchmark, needs to be setup correctly.

Mount the disk partition: The read and write tests would be conducted using files in some partition on the disk. So make sure to mount the disk drive.

Connect to fast usb port: If the disk is connected externally with usb, then make sure to connect it to the fastest usb port available which should be as fast or faster than the rated usb speed of the drive itself.

For example the Seagate Touch 5TB hdd is usb 3.2gen1 so it can be connected to either usb 3.2gen1 port or usb3.2gen2 port. But it cannot be connected to a usb 2.0 port since that would then bottleneck the connection speed and consequently lower the data transfer rates.

Test with Sysbench

So finally we can run tests on the disk drive using sysbench. Open a terminal and navigate to the directory where you would like to perform the test.

Create a dummy directory to conduct tests inside:

[email protected]:/media/acerlight/One Touch$ mkdir sysbenchtest
[email protected]:/media/acerlight/One Touch$ cd sysbenchtest/
ace[email protected]:/media/acerlight/One Touch/sysbenchtest$

No inside this directory sysbenchtest we shall run couple of commands to test the disk io speed.

Sequential IO Tests

This involves reading and writing a large file and noting the maximum data transfer speed. Make sure that the file size is larger than the size of the ram available on the system.

Disk Write Speed: Performing a sequential write test is actually quite simple. Lets create a large 32GB single file and see how long does it take for sysbench to create it.

Here is the commmand:

$ sysbench fileio --file-total-size=32G --file-num=1 prepare
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
1 files, 33554432Kb each, 32768Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
34359738368 bytes written in 314.04 seconds (104.34 MiB/sec).
ace[email protected]:/media/acerlight/One Touch/sysbenchtest$

Note the last line with the speed mentioned at the end:

34359738368 bytes written in 314.04 seconds (104.34 MiB/sec).

So the Sequential Write Speed: 104.34 MiB/s.

Creating multiple files: Instead of creating single large file, you can create multiple files which can also be used to measure the sequential write speed. Here i shall create 5 files with a total size of 5GB (means 1 GB each)

$ sysbench fileio --file-total-size=5G --file-num=5 prepare
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
5 files, 1048576Kb each, 5120Mb total
Creating files for the test...
Extra file open flags: (none)
Creating file test_file.0
Creating file test_file.1
Creating file test_file.2
Creating file test_file.3
Creating file test_file.4
5368709120 bytes written in 49.06 seconds (104.37 MiB/sec).
ace[email protected]:/media/acerlight/One Touch/sysbenchtest$

In this test the Sequential write speed: 104.37 MiB/sec is close to the previous value.

Disk Read Speed: For disk read speed, we shall simply read the 32 GB file created in the earlier step, and sysbench will tell us how long it took to read.

$ sysbench fileio --file-total-size=32G --file-num=1 --file-block-size=16M --file-test-mode=seqrd --time=300 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
1 files, 32GiB each
32GiB total file size
Block size 16MiB
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential read test
Initializing worker threads...
Threads started!
File operations:
    reads/s:                      7.54
    writes/s:                     0.00
    fsyncs/s:                     0.00
Throughput:
    read, MiB/s:                  120.64
    written, MiB/s:               0.00
General statistics:
    total time:                          300.0028s
    total number of events:              2262
Latency (ms):
         min:                                  120.63
         avg:                                  132.62
         max:                                  435.78
         95th percentile:                      150.29
         sum:                               299994.24
Threads fairness:
    events (avg/stddev):           2262.0000/0.00
    execution time (avg/stddev):   299.9942/0.00
ace[email protected]:/media/acerlight/One Touch/sysbenchtest$

Note the Throughput section which mentions the read speed:

Throughput:
    read, MiB/s:                  120.64
    written, MiB/s:               0.00

Sequential Read Speed: 120.64 MiB/s.

Random IO Test

Now lets perform random read-write tests. The idea is to operate on multiple files in different orders so that the disk has to execute open/close/seek operations frequently and handle latency in file operations.

Random Write: If you run the command without many options it will simply perform a random write test by creating and writing 128 files of size 16 MiB each (total: 2GiB).

Yes the following command is good for doing a random write test. You do not need to run prepare before this, as it will create files on its own.

sysbench fileio --file-test-mode=seqwr --time=600 run

Output:

[email protected]:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio --file-test-mode=seqwr --time=600 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 16MiB each
2GiB total file size
Block size 16KiB
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential write (creation) test
Initializing worker threads...
Threads started!
File operations:
    reads/s:                      0.00
    writes/s:                     361.66
    fsyncs/s:                     463.06
Throughput:
    read, MiB/s:                  0.00
    written, MiB/s:               5.65
General statistics:
    total time:                          600.2893s
    total number of events:              494944
Latency (ms):
         min:                                    0.00
         avg:                                    1.21
         max:                                  318.99
         95th percentile:                        1.44
         sum:                               599504.37
Threads fairness:
    events (avg/stddev):           494944.0000/0.00
    execution time (avg/stddev):   599.5044/0.00
ace[email protected]:/media/acerlight/One Touch/sysbenchtest$

Note the Throughtput section for the write speed

Throughput:
    read, MiB/s:                  0.00
    written, MiB/s:               5.65

Random write speed: 5.65 MiB/s

Monitoring in background: If you want to monitor the disk io activity in background, use htop or iotop commands with suitable options. Here are some examples

Sysbench Disk Seqwr Htop Monitoring

Sysbench Disk Seqwr Htop Monitoring

Sysbench Disk Seqwr Iotop Monitoring

Sysbench Disk Seqwr Iotop Monitoring

Even though the test mode is --file-test-mode=seqwr (sequential write), this test with its settings and mode of operation effectively emulates a random write test.

Random IO tests heavily depend on the number of files and size of the individual files. If there are lots of small sized files, the average disk io speed will be low since there will be lots of "disk seek" operations involved for each file, which introduce latency in the overall operation.

Random Read: Now its time to run random read tests. We can simply read the files created above in sequence and it would represent values equivalent to random read operations.

Funny values: If you try to run seqrd on the files already created in the previous step you will get illogically large values for the read speed. Here is an example

[email protected]:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio --file-test-mode=seqrd --time=600 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 16MiB each
2GiB total file size
Block size 16KiB
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential read test
Initializing worker threads...
Threads started!
File operations:
    reads/s:                      531941.26
    writes/s:                     0.00
    fsyncs/s:                     0.00
Throughput:
    read, MiB/s:                  8311.58
    written, MiB/s:               0.00
General statistics:
    total time:                          600.0001s
    total number of events:              319165574
Latency (ms):
         min:                                    0.00
         avg:                                    0.00
         max:                                    1.31
         95th percentile:                        0.00
         sum:                               563384.63
Threads fairness:
    events (avg/stddev):           319165574.0000/0.00
    execution time (avg/stddev):   563.3846/0.00
ace[email protected]:/media/acerlight/One Touch/sysbenchtest$

Note the read speed: 8311.58 MiB/s
This is simply nonsense!

Now lets try to see why this happens.

Cleanup the Cache: Important

Right after the random write test, we can simply run the rndrd (random read test). But we must make sure to clean the cache. The commands should be in the following order.

sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"
sysbench fileio --file-test-mode=rndrd --time=600 run

Note: If you had created custom sized files in the write step, then you must specify that in the read command as well.

The output of random read could look something like this:

[email protected]:/media/acerlight/One Touch/sysbenchtest$ sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"
[email protected]:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio --file-test-mode=rndrd --time=600 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 16MiB each
2GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Initializing worker threads...
Threads started!
File operations:
    reads/s:                      92.82
    writes/s:                     0.00
    fsyncs/s:                     0.00
Throughput:
    read, MiB/s:                  1.45
    written, MiB/s:               0.00
General statistics:
    total time:                          600.0136s
    total number of events:              55692
Latency (ms):
         min:                                    0.00
         avg:                                   10.77
         max:                                   93.91
         95th percentile:                       21.89
         sum:                               599866.10
Threads fairness:
    events (avg/stddev):           55692.0000/0.00
    execution time (avg/stddev):   599.8661/0.00
ace[email protected]:/media/acerlight/One Touch/sysbenchtest$

Note the Throughput section:

Throughput:
    read, MiB/s:                  1.45
    written, MiB/s:               0.00

Random read speed: 1.45 MiB/s. This is low but is expected since its an hdd and we are doing random reads.

Before doing read tests, always make sure to clear the cache first, otherwise sysbench would read the files from cache and not the drive. This will be crazy fast and the reported read speed would be unrealistically high.

Cleanup Test Files

Sysbench creates a lot of dummy files for read write testing. The files contain dummy garbage values and they are just used to perform read and write operations. Larger fewer files are useful for sequential operation tests, whereas lots of small sized files are used for random read-write tests.

These look somewhat like this:

[email protected]:/media/acerlight/One Touch/sysbenchtest$ ls -la
total 2097664
drwxr-xr-x  2 acerlight acerlight   262144 May 10 09:12 .
drwxr-xr-x 13 acerlight acerlight   262144 May 10 11:38 ..
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.0
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.1
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.10
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.100
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.101
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.102
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.103
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.104
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.105
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.106
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.107
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.108
-rwxr-xr-x  1 acerlight acerlight 16777216 May 10 09:26 test_file.109
...

You can change the number of files and size of each by using suitable options.

After you are done with the tests, remove the test files by using the rm command or cleanup option with sysbench. This should be done in the same directory as where the prepare or write commands were executed to create the test files.

acerligh[email protected]:/media/acerlight/One Touch/sysbenchtest$ rm *
[email protected]:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio cleanup

Summary

So now that we have run the tests, lets create a simple summary table displaying all the values.

Device Sequential Write Speed (MiB/s) Seq. Read Speed (MiB/s) Random Write (MiB/s) Random Read (MiB/s)
Seagate One Touch 5TB HDD 105 120 6 1.5

The random read-write speeds are way slower than the sequential ones. This is specially because its a hard drive and the location of the data is physically dependant on the disk tracks and sectors.

Other commands

There are other commands including fio and dd that can be used to perform similar read write tests on disks to benchmark it performance. Check out these articles to learn more:

Test read/write speed of usb and ssd drives with dd command on Linux

Conclusion

Testing the read write speed of your disk drive is one of the crucial benchmarks for performance of your system. Slow disks can affect the overall performance of the system.

Applications will load slow, since files are read when launching an application for first time. Operations involving editing files will run slow, for instance database operations. Database operations are one of the critical applications that need as fast disk io as possible.

SSDs and NVMe drives are now becoming the standard for server and desktop machines as they are much more faster compared to hdds with read-write speeds in GiBs. This makes them very suitable for use case scenarios that involve working with large files or lots of files.

Let us know in the comments below.

About Silver Moon

A Tech Enthusiast, Blogger, Linux Fan and a Software Developer. Writes about Computer hardware, Linux and Open Source software and coding in Python, Php and Javascript. He can be reached at [email protected].

Leave a Reply

Your email address will not be published. Required fields are marked *