Whether it’s a server, or a PC for work, what usually limits performances is disk speed. Even if using SSDs, their speed is not yet comparable to that of RAM and CPU.
There are different tools with or without a graphical interface, written for testing disks speed. There are also people who use
, for example:
dd if=/dev/zero of=test_file bs=64k count=16k conv=fdatasync
However, in our opinion dd is the worst software for benchmarking I/O performance.
In fact:
- it is a single-threaded, sequential-write test. Of course, if running a web server, services do not do long-running sequential writes, and use more than one thread
- it writes a small amount of data, so the result can be influenced by caching or by RAID’s controller
- it executes for just a few seconds, and everyone knows that in this way it’s not possible to have consistent results
- there are no reading speed tests
All these points just lead to one conclusion: better to use anything else. For disk benchmarking there are two kind of parameters that give a complete overview: IOPS (I/O Per Second) and latency. This tutorial explains how to measure IOPS with
, and disk latency with
on a RHEL 7 system.
Install fio
First of all, install the EPEL repository:
# wget https://mirrors.n-ix.net/fedora-epel/epel-release-latest-7.noarch.rpm # yum localinstall epel-release-latest-7.noarch.rpm
Next, install fio with yum:
# yum install fio
Testing IOPS with fio
RW Performance
The first test is for measuring random read/write performances. In a terminal, execute the following command:
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
During the test, the terminal window will display an output like the following one:
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64 fio-2.2.8 Starting 1 process test: Laying out IO file(s) (1 file(s) / 4096MB) Jobs: 1 (f=1): [m(1)] [0.1% done] [447KB/131KB/0KB /s] [111/32/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [383KB/147KB/0KB /s] [95/36/0 iops] [eta 01h:4Jobs: 1 (f=1): [m(1)] [0.1% done] [456KB/184KB/0KB /s] [114/46/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [624KB/188KB/0KB /s] [156/47/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [443KB/115KB/0KB /s] [110/28/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.1% done] [515KB/95KB/0KB /s] [128/23/0 iops] [eta 01h:4Jobs: 1 (f=1): [m(1)] [0.1% done] [475KB/163KB/0KB /s] [118/40/0 iops] [eta 01h:Jobs: 1 (f=1): [m(1)] [0.2% done] [451KB/127KB/0KB /s] [112/31/0 iops]
So, the program will create a 4GB file (
), and perform 4KB reads and writes using three reads for every write ratio (75%/25%, as specified with option
), split within the file, with 64 operations running at a time. The RW ratio can be adjusted for simulating various usage scenarios.
At the end, it will display the final results:
test: (groupid=0, jobs=1): err= 0: pid=4760: Thu Mar 2 13:23:28 2017
read : io=7884.0KB, bw=864925B/s, iops=211, runt= 9334msec
write: io=2356.0KB, bw=258468B/s, iops=63, runt= 9334msec
cpu : usr=0.46%, sys=2.35%, ctx=2289, majf=0, minf=29
IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=0.6%, 32=1.2%, >=64=97.5%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=1971/w=589/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=7884KB, aggrb=844KB/s, minb=844KB/s, maxb=844KB/s, mint=9334msec, maxt=9334msec
WRITE: io=2356KB, aggrb=252KB/s, minb=252KB/s, maxb=252KB/s, mint=9334msec, maxt=9334msec
Disk stats (read/write):
dm-2: ios=1971/589, merge=0/0, ticks=454568/120101, in_queue=581406, util=98.44%, aggrios=1788/574, aggrmerge=182/15, aggrticks=425947/119120, aggrin_queue=545252, aggrutil=98.48%
sda: ios=1788/574, merge=182/15, ticks=425947/119120, in_queue=545252, util=98.48%
Note from the author: I ran fio on my laptop, so the last output was obtained running the test with a 10MB file; as can be seen above, the 4GB option would have taken more than 1 hour.
Random read performance
In this case, the command is:
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read.fio --bs=4k --iodepth=64 --size=4G --readwrite=randread
The output will be similar to the RW case, just specialized in the read case.
Random write performance
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randwrite
As above, in random write case.
Latency measures with IOPing
As stated in the introduction, the second part of a benchmark is the latency measurement. To accomplish this task, install IOPing, also available in the EPEL repository.
# yum install ioping
Execute it:
# ioping -c 100 .
The
option is the number request ioping will make. The program takes also as argument the file and/or device to check. In this case, the actual working directory. Program output is:
4 KiB <<< . (xfs /dev/dm-2): request=1 time=16.3 ms (warmup) 4 KiB <<< . (xfs /dev/dm-2): request=2 time=253.3 us 4 KiB <<< . (xfs /dev/dm-2): request=3 time=284.0 ms ... 4 KiB <<< . (xfs /dev/dm-2): request=96 time=175.6 us (fast) 4 KiB <<< . (xfs /dev/dm-2): request=97 time=258.7 us (fast) 4 KiB <<< . (xfs /dev/dm-2): request=98 time=277.6 us (fast) 4 KiB <<< . (xfs /dev/dm-2): request=99 time=242.3 us (fast) 4 KiB <<< . (xfs /dev/dm-2): request=100 time=36.1 ms (fast) --- . (xfs /dev/dm-2) ioping statistics --- 99 requests completed in 3.99 s, 396 KiB read, 24 iops, 99.3 KiB/s generated 100 requests in 1.65 min, 400 KiB, 1 iops, 4.04 KiB/s min/avg/max/mdev = 163.5 us / 40.3 ms / 760.0 ms / 118.5 ms
Last line shows the latency measures of the disk.
Conclusion:
So here there are two useful tools which can give information about the status of disk speed and latency, with very clear outputs, and customizable for various test cases.
