Difference between revisions of "Io tests"

From UVOO Tech Wiki
Jump to navigation Jump to search
Line 54: Line 54:
 
Run status group 0 (all jobs):
 
Run status group 0 (all jobs):
 
   READ: bw=69.2MiB/s (72.6MB/s), 69.2MiB/s-69.2MiB/s (72.6MB/s-72.6MB/s), io=10.0GiB (10.7GB), run=147882-147882msec
 
   READ: bw=69.2MiB/s (72.6MB/s), 69.2MiB/s-69.2MiB/s (72.6MB/s-72.6MB/s), io=10.0GiB (10.7GB), run=147882-147882msec
 +
```
 +
 +
 +
# Cheat Sheets
 +
 +
## One
 +
 +
```
 +
-----------------------------------------------------------------------------------------------------
 +
Read Test
 +
fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting
 +
-----------------------------------------------------------------------------------------------------
 +
writes a total 2GB files [4 jobs x 512 MB = 2GB] running 4 processes at a time:
 +
 +
fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting
 +
-----------------------------------------------------------------------------------------------------
 +
Read Write Performance Test
 +
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
 +
-----------------------------------------------------------------------------------------------------
 +
Sequential Reads – Async mode – 8K block size – Direct IO – 100% Reads
 +
 +
fio --name=seqread --rw=read --direct=1 --ioengine=libaio --bs=8k --numjobs=8 --size=1G --runtime=600  --group_reporting
 +
-----------------------------------------------------------------------------------------------------
 +
Sequential Writes – Async mode – 32K block size – Direct IO – 100% Writes
 +
 +
fio --name=seqwrite --rw=write --direct=1 --ioengine=libaio --bs=32k --numjobs=4 --size=2G --runtime=600 --group_reporting
 +
-----------------------------------------------------------------------------------------------------
 +
Random Reads – Async mode – 8K block size – Direct IO – 100% Reads
 +
 +
fio --name=randread --rw=randread --direct=1 --ioengine=libaio --bs=8k --numjobs=16 --size=1G --runtime=600 --group_reporting
 +
-----------------------------------------------------------------------------------------------------
 +
Random Writes – Async mode – 64K block size – Direct IO – 100% Writes
 +
 +
fio --name=randwrite --rw=randwrite --direct=1 --ioengine=libaio --bs=64k --numjobs=8 --size=512m --runtime=600 --group_reporting
 +
-----------------------------------------------------------------------------------------------------
 +
Random Read/Writes – Async mode – 16K block size – Direct IO – 90% Reads/10% Writes
 +
 +
fio --name=randrw --rw=randrw --direct=1 --ioengine=libaio --bs=16k --numjobs=8 --rwmixread=90 --size=1G --runtime=600 --group_reporting
 +
-----------------------------------------------------------------------------------------------------
 +
creates 8 files (numjobs=8) each with size 512MB (size) at 64K block size (bs=64k) and will perform random read/write (rw=randrw) with the mixed workload of 70% reads and 30% writes.
 +
The job will run for full 5 minutes (runtime=300 & time_based) even if the files were created and read/written.
 +
 +
fio --name=randrw --ioengine=libaio --iodepth=1 --rw=randrw --bs=64k --direct=1 --size=512m --numjobs=8 --runtime=300 --group_reporting --time_based --rwmixread=70
 +
 +
-----------------------------------------------------------------------------------------------------
 +
compare disk performance with a simple 3:1 4K read/write test
 +
creates a 4 GB file and perform 4KB reads and writes using a 75%/25% split within the file, with 64 operations running at a time. The 3:1 ratio represents a typical database.
 +
 +
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
 +
-----------------------------------------------------------------------------------------------------
 +
Random read performance
 +
 +
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randread
 +
-----------------------------------------------------------------------------------------------------
 +
Random write performance
 +
 +
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randwrite
 +
-----------------------------------------------------------------------------------------------------
 +
-----------------------------------------------------------------------------------------------------
 +
 +
-----------------------------------------------------------------------------------------------------
 
```
 
```

Revision as of 16:38, 25 November 2023

apt install fio 

#!/usr/bin/env bash

set -o errexit

echo $(date):Random read
fio --filename=test --sync=1 --rw=randread --bs=4k --numjobs=1 \
  --iodepth=4 --group_reporting --name=test --filesize=10G --runtime=300 && rm test

echo $(date):Random write
fio --filename=test --sync=1 --rw=randwrite --bs=4k --numjobs=1 \
  --iodepth=4 --group_reporting --name=test --filesize=10G --runtime=300 && rm test

echo $(date):Sequential read
fio --filename=test --sync=1 --rw=read --bs=4k --numjobs=1 \
  --iodepth=4 --group_reporting --name=test --filesize=10G --runtime=300 && rm test

echo $(date):Sequential write
fio --filename=test --sync=1 --rw=write --bs=4k --numjobs=1 \
  --iodepth=4 --group_reporting --name=test --filesize=10G --runtime=300 && rm test

Output example

Sat Nov 25 16:25:47 UTC 2023:Random read
test: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=4
fio-3.28
Starting 1 process
test: Laying out IO file (1 file / 10240MiB)
Jobs: 1 (f=1): [r(1)][98.7%][r=130MiB/s][r=33.3k IOPS][eta 00m:02s]
test: (groupid=0, jobs=1): err= 0: pid=1894: Sat Nov 25 16:29:04 2023
  read: IOPS=17.7k, BW=69.2MiB/s (72.6MB/s)(10.0GiB/147882msec)
    clat (usec): min=3, max=9506, avg=55.42, stdev=23.89
     lat (usec): min=3, max=9506, avg=55.49, stdev=23.91
    clat percentiles (usec):
     |  1.00th=[    6],  5.00th=[    7], 10.00th=[   45], 20.00th=[   53],
     | 30.00th=[   57], 40.00th=[   58], 50.00th=[   59], 60.00th=[   62],
     | 70.00th=[   64], 80.00th=[   66], 90.00th=[   68], 95.00th=[   70],
     | 99.00th=[   74], 99.50th=[   75], 99.90th=[   80], 99.95th=[   83],
     | 99.99th=[ 1614]
   bw (  KiB/s): min=64224, max=175456, per=99.56%, avg=70592.32, stdev=9296.71, samples=295
   iops        : min=16056, max=43864, avg=17648.03, stdev=2324.16, samples=295
  lat (usec)   : 4=0.06%, 10=8.27%, 20=0.02%, 50=8.75%, 100=82.87%
  lat (usec)   : 250=0.01%, 500=0.01%, 750=0.01%
  lat (msec)   : 2=0.01%, 10=0.01%
  cpu          : usr=3.31%, sys=96.66%, ctx=822, majf=14, minf=15
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=2621440,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=4

Run status group 0 (all jobs):
   READ: bw=69.2MiB/s (72.6MB/s), 69.2MiB/s-69.2MiB/s (72.6MB/s-72.6MB/s), io=10.0GiB (10.7GB), run=147882-147882msec

Cheat Sheets

One

-----------------------------------------------------------------------------------------------------
 Read Test
 fio --name=randread --ioengine=libaio --iodepth=16 --rw=randread --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting
-----------------------------------------------------------------------------------------------------
writes a total 2GB files [4 jobs x 512 MB = 2GB] running 4 processes at a time:

fio --name=randwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=4k --direct=0 --size=512M --numjobs=4 --runtime=240 --group_reporting
-----------------------------------------------------------------------------------------------------
 Read Write Performance Test
 fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
 -----------------------------------------------------------------------------------------------------
Sequential Reads – Async mode – 8K block size – Direct IO – 100% Reads

fio --name=seqread --rw=read --direct=1 --ioengine=libaio --bs=8k --numjobs=8 --size=1G --runtime=600  --group_reporting
-----------------------------------------------------------------------------------------------------
Sequential Writes – Async mode – 32K block size – Direct IO – 100% Writes

fio --name=seqwrite --rw=write --direct=1 --ioengine=libaio --bs=32k --numjobs=4 --size=2G --runtime=600 --group_reporting
-----------------------------------------------------------------------------------------------------
Random Reads – Async mode – 8K block size – Direct IO – 100% Reads

fio --name=randread --rw=randread --direct=1 --ioengine=libaio --bs=8k --numjobs=16 --size=1G --runtime=600 --group_reporting
-----------------------------------------------------------------------------------------------------
Random Writes – Async mode – 64K block size – Direct IO – 100% Writes

fio --name=randwrite --rw=randwrite --direct=1 --ioengine=libaio --bs=64k --numjobs=8 --size=512m --runtime=600 --group_reporting
-----------------------------------------------------------------------------------------------------
Random Read/Writes – Async mode – 16K block size – Direct IO – 90% Reads/10% Writes

fio --name=randrw --rw=randrw --direct=1 --ioengine=libaio --bs=16k --numjobs=8 --rwmixread=90 --size=1G --runtime=600 --group_reporting
-----------------------------------------------------------------------------------------------------
 creates 8 files (numjobs=8) each with size 512MB (size) at 64K block size (bs=64k) and will perform random read/write (rw=randrw) with the mixed workload of 70% reads and 30% writes. 
 The job will run for full 5 minutes (runtime=300 & time_based) even if the files were created and read/written.

 fio --name=randrw --ioengine=libaio --iodepth=1 --rw=randrw --bs=64k --direct=1 --size=512m --numjobs=8 --runtime=300 --group_reporting --time_based --rwmixread=70

-----------------------------------------------------------------------------------------------------
compare disk performance with a simple 3:1 4K read/write test
creates a 4 GB file and perform 4KB reads and writes using a 75%/25% split within the file, with 64 operations running at a time. The 3:1 ratio represents a typical database.

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75
-----------------------------------------------------------------------------------------------------
Random read performance

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randread
-----------------------------------------------------------------------------------------------------
Random write performance

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randwrite
-----------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------

-----------------------------------------------------------------------------------------------------
Retrieved from "https://tech.uvoo.io/index.php?title=Io_tests&oldid=4719"