Sysbench is a command line utility program available on linux that can be used to conduct different types of benchmark tests to measure performance of disk io, memory, cpu and mysql database.
In another article we learned how to use sysbench to test ram read/write speed.
In this particular article we take a quick look at how we can use the sysbench utility program to benchmark disk io speed.
For the sake of the this experiment we shall be used different disk drives like seagate external hdd, samsung external ssd, usb flash drives etc.
Install sysbench
The first thing that you need to do is to install sysbench. It is available in the repositories of most linux distros.
Ubuntu
sudo apt install sysbench
Setup the drive
The disk drive that you want to benchmark, needs to be setup correctly.
Mount the disk partition: The read and write tests would be conducted using files in some partition on the disk. So make sure to mount the disk drive.
Connect to fast usb port: If the disk is connected externally with usb, then make sure to connect it to the fastest usb port available which should be as fast or faster than the rated usb speed of the drive itself.
For example the Seagate Touch 5TB hdd is usb 3.2gen1 so it can be connected to either usb 3.2gen1 port or usb3.2gen2 port. But it cannot be connected to a usb 2.0 port since that would then bottleneck the connection speed and consequently lower the data transfer rates.
Test with Sysbench
So finally we can run tests on the disk drive using sysbench. Open a terminal and navigate to the directory where you would like to perform the test.
Create a dummy directory to conduct tests inside:
acerlight@acerlight-laptop:/media/acerlight/One Touch$ mkdir sysbenchtest acerlight@acerlight-laptop:/media/acerlight/One Touch$ cd sysbenchtest/ acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$
No inside this directory sysbenchtest we shall run couple of commands to test the disk io speed.
Sequential IO Tests
This involves reading and writing a large file and noting the maximum data transfer speed. Make sure that the file size is larger than the size of the ram available on the system.
Disk Write Speed: Performing a sequential write test is actually quite simple. Lets create a large 32GB single file and see how long does it take for sysbench to create it.
Here is the commmand:
$ sysbench fileio --file-total-size=32G --file-num=1 prepare sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3) 1 files, 33554432Kb each, 32768Mb total Creating files for the test... Extra file open flags: (none) Creating file test_file.0 34359738368 bytes written in 314.04 seconds (104.34 MiB/sec). acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$
Note the last line with the speed mentioned at the end:
34359738368 bytes written in 314.04 seconds (104.34 MiB/sec).
So the Sequential Write Speed: 104.34 MiB/s.
Creating multiple files: Instead of creating single large file, you can create multiple files which can also be used to measure the sequential write speed. Here i shall create 5 files with a total size of 5GB (means 1 GB each)
$ sysbench fileio --file-total-size=5G --file-num=5 prepare sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3) 5 files, 1048576Kb each, 5120Mb total Creating files for the test... Extra file open flags: (none) Creating file test_file.0 Creating file test_file.1 Creating file test_file.2 Creating file test_file.3 Creating file test_file.4 5368709120 bytes written in 49.06 seconds (104.37 MiB/sec). acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$
In this test the Sequential write speed: 104.37 MiB/sec is close to the previous value.
Disk Read Speed: For disk read speed, we shall simply read the 32 GB file created in the earlier step, and sysbench will tell us how long it took to read.
$ sysbench fileio --file-total-size=32G --file-num=1 --file-block-size=16M --file-test-mode=seqrd --time=300 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
1 files, 32GiB each
32GiB total file size
Block size 16MiB
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential read test
Initializing worker threads...
Threads started!
File operations:
reads/s: 7.54
writes/s: 0.00
fsyncs/s: 0.00
Throughput:
read, MiB/s: 120.64
written, MiB/s: 0.00
General statistics:
total time: 300.0028s
total number of events: 2262
Latency (ms):
min: 120.63
avg: 132.62
max: 435.78
95th percentile: 150.29
sum: 299994.24
Threads fairness:
events (avg/stddev): 2262.0000/0.00
execution time (avg/stddev): 299.9942/0.00
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$
Note the Throughput section which mentions the read speed:
Throughput:
read, MiB/s: 120.64
written, MiB/s: 0.00
Sequential Read Speed: 120.64 MiB/s.
Random IO Test
Now lets perform random read-write tests. The idea is to operate on multiple files in different orders so that the disk has to execute open/close/seek operations frequently and handle latency in file operations.
Random Write: If you run the command without many options it will simply perform a random write test by creating and writing 128 files of size 16 MiB each (total: 2GiB).
Yes the following command is good for doing a random write test. You do not need to run prepare before this, as it will create files on its own.
sysbench fileio --file-test-mode=seqwr --time=600 run
Output:
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio --file-test-mode=seqwr --time=600 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 16MiB each
2GiB total file size
Block size 16KiB
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential write (creation) test
Initializing worker threads...
Threads started!
File operations:
reads/s: 0.00
writes/s: 361.66
fsyncs/s: 463.06
Throughput:
read, MiB/s: 0.00
written, MiB/s: 5.65
General statistics:
total time: 600.2893s
total number of events: 494944
Latency (ms):
min: 0.00
avg: 1.21
max: 318.99
95th percentile: 1.44
sum: 599504.37
Threads fairness:
events (avg/stddev): 494944.0000/0.00
execution time (avg/stddev): 599.5044/0.00
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$
Note the Throughtput section for the write speed
Throughput:
read, MiB/s: 0.00
written, MiB/s: 5.65
Random write speed: 5.65 MiB/s
Monitoring in background: If you want to monitor the disk io activity in background, use htop or iotop commands with suitable options. Here are some examples
Sysbench Disk Seqwr Htop Monitoring
Sysbench Disk Seqwr Iotop Monitoring
Even though the test mode is --file-test-mode=seqwr (sequential write), this test with its settings and mode of operation effectively emulates a random write test.
Random IO tests heavily depend on the number of files and size of the individual files. If there are lots of small sized files, the average disk io speed will be low since there will be lots of "disk seek" operations involved for each file, which introduce latency in the overall operation.
Random Read: Now its time to run random read tests. We can simply read the files created above in sequence and it would represent values equivalent to random read operations.
Funny values: If you try to run seqrd on the files already created in the previous step you will get illogically large values for the read speed. Here is an example
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio --file-test-mode=seqrd --time=600 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 16MiB each
2GiB total file size
Block size 16KiB
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing sequential read test
Initializing worker threads...
Threads started!
File operations:
reads/s: 531941.26
writes/s: 0.00
fsyncs/s: 0.00
Throughput:
read, MiB/s: 8311.58
written, MiB/s: 0.00
General statistics:
total time: 600.0001s
total number of events: 319165574
Latency (ms):
min: 0.00
avg: 0.00
max: 1.31
95th percentile: 0.00
sum: 563384.63
Threads fairness:
events (avg/stddev): 319165574.0000/0.00
execution time (avg/stddev): 563.3846/0.00
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$
Note the read speed: 8311.58 MiB/s
This is simply nonsense!
Now lets try to see why this happens.
Cleanup the Cache: Important
Right after the random write test, we can simply run the rndrd (random read test). But we must make sure to clean the cache. The commands should be in the following order.
sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches" sysbench fileio --file-test-mode=rndrd --time=600 run
Note: If you had created custom sized files in the write step, then you must specify that in the read command as well.
The output of random read could look something like this:
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$ sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches"
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio --file-test-mode=rndrd --time=600 run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
Extra file open flags: (none)
128 files, 16MiB each
2GiB total file size
Block size 16KiB
Number of IO requests: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random read test
Initializing worker threads...
Threads started!
File operations:
reads/s: 92.82
writes/s: 0.00
fsyncs/s: 0.00
Throughput:
read, MiB/s: 1.45
written, MiB/s: 0.00
General statistics:
total time: 600.0136s
total number of events: 55692
Latency (ms):
min: 0.00
avg: 10.77
max: 93.91
95th percentile: 21.89
sum: 599866.10
Threads fairness:
events (avg/stddev): 55692.0000/0.00
execution time (avg/stddev): 599.8661/0.00
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$
Note the Throughput section:
Throughput:
read, MiB/s: 1.45
written, MiB/s: 0.00
Random read speed: 1.45 MiB/s. This is low but is expected since its an hdd and we are doing random reads.
Before doing read tests, always make sure to clear the cache first, otherwise sysbench would read the files from cache and not the drive. This will be crazy fast and the reported read speed would be unrealistically high.
Cleanup Test Files
Sysbench creates a lot of dummy files for read write testing. The files contain dummy garbage values and they are just used to perform read and write operations. Larger fewer files are useful for sequential operation tests, whereas lots of small sized files are used for random read-write tests.
These look somewhat like this:
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$ ls -la total 2097664 drwxr-xr-x 2 acerlight acerlight 262144 May 10 09:12 . drwxr-xr-x 13 acerlight acerlight 262144 May 10 11:38 .. -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.0 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.1 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.10 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.100 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.101 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.102 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.103 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.104 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.105 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.106 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.107 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.108 -rwxr-xr-x 1 acerlight acerlight 16777216 May 10 09:26 test_file.109 ...
You can change the number of files and size of each by using suitable options.
After you are done with the tests, remove the test files by using the rm command or cleanup option with sysbench. This should be done in the same directory as where the prepare or write commands were executed to create the test files.
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$ rm *
acerlight@acerlight-laptop:/media/acerlight/One Touch/sysbenchtest$ sysbench fileio cleanup
Summary
So now that we have run the tests, lets create a simple summary table displaying all the values.
| Device | Sequential Write Speed (MiB/s) | Seq. Read Speed (MiB/s) | Random Write (MiB/s) | Random Read (MiB/s) |
| Seagate One Touch 5TB HDD | 105 | 120 | 6 | 1.5 |
The random read-write speeds are way slower than the sequential ones. This is specially because its a hard drive and the location of the data is physically dependant on the disk tracks and sectors.
Other commands
There are other commands including fio and dd that can be used to perform similar read write tests on disks to benchmark it performance. Check out these articles to learn more:
Test read/write speed of usb and ssd drives with dd command on LinuxConclusion
Testing the read write speed of your disk drive is one of the crucial benchmarks for performance of your system. Slow disks can affect the overall performance of the system.
Applications will load slow, since files are read when launching an application for first time. Operations involving editing files will run slow, for instance database operations. Database operations are one of the critical applications that need as fast disk io as possible.
SSDs and NVMe drives are now becoming the standard for server and desktop machines as they are much more faster compared to hdds with read-write speeds in GiBs. This makes them very suitable for use case scenarios that involve working with large files or lots of files.
Let us know in the comments below.
