Some Linux performance testing commands

This short tutorial mentions some Linux performance testing commands to measure server/VPS performance. This will be added gradually as soon as I have more tools to test 🙂

1. Test disk IO:

[bash]$ dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync
16384+0 records in
16384+0 records out
1073741824 bytes (1.1 GB) copied, 15.1643 s, 70.8 MB/s[/bash]

2. Test disk IO with ioping

First, we need to install ioping (current version is 0.8):

[bash]wget https://github.com/koct9i/ioping/archive/v1.0.tar.gz
tar zxvf v1.0.tar.gz
cd ioping-1.0
make
make install
[/bash]

Or you can simply download the package for CentOS 7 at http://dl.fedoraproject.org/pub/epel/7/x86_64/i/ and install it with

[bash]rpm -Uvh http://dl.fedoraproject.org/pub/epel/7/x86_64/i/ioping-1.0-1.el7.x86_64.rpm[/bash]

. After that, we can use ioping instead of heavy / abusing dd command:

Measure disk seek rate (iops, avg):[bash][root@flkvm ioping-0.8]# ioping -R /dev/sda
— /dev/sda (device 40.0 GiB) ioping statistics —
66 requests completed in 3.0 s, 23 iops, 92.1 KiB/s
min/avg/max/mdev = 3.4 ms / 43.4 ms / 259.7 ms / 55.3 ms[/bash]
[bash][root@mynvme ~]# ioping -R /dev/vda
— /dev/vda (block device 25 GiB) ioping statistics —
29.8 k requests completed in 2.84 s, 116.5 MiB read, 10.5 k iops, 40.9 MiB/s
generated 29.8 k requests in 3.00 s, 116.5 MiB, 9.94 k iops, 38.8 MiB/s
min/avg/max/mdev = 75.5 us / 95.4 us / 1.41 ms / 15.4 us[/bash]
Measure disk sequential speed (MiB/s):[bash][root@flkvm ioping-0.8]# ioping -RL /dev/sda
— /dev/sda (device 40.0 GiB) ioping statistics —
60 requests completed in 3.0 s, 23 iops, 5.8 MiB/s
min/avg/max/mdev = 2.1 ms / 43.1 ms / 269.2 ms / 61.1 ms[/bash]
[bash][root@mynvme ~]# ioping -RL /dev/vda
— /dev/vda (block device 25 GiB) ioping statistics —
10.0 k requests completed in 2.88 s, 2.45 GiB read, 3.48 k iops, 869.9 MiB/s
generated 10.0 k requests in 3.00 s, 2.45 GiB, 3.34 k iops, 835.5 MiB/s
min/avg/max/mdev = 186.5 us / 287.4 us / 10.8 ms / 214.1 us[/bash]

3. Test with pre-built script

[bash][root@bluevm ~]# wget https://freevps.us/bench.sh -O – -o /dev/null|bash
CPU model : Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz
Number of cores : 7
CPU frequency : 500.019 MHz
Total amount of ram : 128 MB
Total amount of swap : 128 MB
System uptime : 26 min,
Download speed from CacheFly: 30.8MB/s
Download speed from Coloat, Atlanta GA: 33.7MB/s
Download speed from Softlayer, Dallas, TX: 16.5MB/s
Download speed from Linode, Tokyo, JP: 6.95MB/s
Download speed from i3d.net, NL: 9.05MB/s
Download speed from Leaseweb, Haarlem, NL: 14.0MB/s
Download speed from Softlayer, Singapore: 4.69MB/s
Download speed from Softlayer, Seattle, WA: 11.6MB/s
Download speed from Softlayer, San Jose, CA: 16.5MB/s
Download speed from Softlayer, Washington, DC: 55.4MB/s
I/O speed : 504 MB/s[/bash]

[bash]root@eubackup ~ # wget tienle.com/networktest.sh -O – -o /dev/null|bash
CPU model : Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
—- Number of cores : 8
—- CPU frequency : 1600.000 MHz
Total amount of RAM : 15955 MB
Total amount of SWAP : 8191 MB
System UPTIME: 183 days, 2:53,
Download speed from CacheFly: 60.9MB/s

==============================================================
# America Test #
==============================================================

Download speed from Coloat, Atlanta GA, USA: 13.8MB/s
Download speed from Softlayer, Dallas, TX, USA: 12.8MB/s
Download speed from Vultr, Los Angeles, CA, USA: 6.03MB/s
Download speed from Vultr, Silicon Valley, CA, USA: 479KB/s
Download speed from Softlayer, San Jose, CA, USA: 9.62MB/s
Download speed from Softlayer, Chicago, IL, USA: 6.59MB/s
Download speed from Softlayer, Seattle, WA, USA: 9.70MB/s
Download speed from Softlayer, Washington, DC, USA: 2.39MB/s
Download speed from Vultr, New Jersey, USA: 14.4MB/s
Download speed from OVH, Beauharnois, Canada: 11.5MB/s
Download speed from Azure, Sao Paulo, Brazil: 6.12MB/s

==============================================================
# EU and UK Test #
==============================================================

Download speed from Linode, London, UK: 63.8MB/s
Download speed from Linode, Frankfurt, DE: 17.4MB/s
Download speed from Leaseweb, Haarlem, NL, West EU: 61.2MB/s
Download speed from i3d.net, Rotterdam, NL, West EU: 36.8MB/s
Download speed from i3d.net, Ireland, North EU: 45.1MB/s
Download speed from OVH, Roubaix, FR: 56.4MB/s
Download speed from Prometeus, Milan, IT: 9.08MB/s

==============================================================
# Oceania & Asia Test #
==============================================================

Download speed from Azure, New South Wales, Australia East: 1.65MB/s
Download speed from Azure, Victoria, Australia Southeast: 1.20MB/s
Download speed from Softlayer, Singapore: 4.56MB/s
Download speed from Linode, Singapore: 7.41MB/s
Download speed from Linode, Tokyo, JP: 7.12MB/s
Download speed from Azure, Hong Kong, CN: 4.88MB/s
Download speed from Azure, Shanghai, CN:
Download speed from Azure, Beijing, CN:

==============================================================
# Disk IO Test #
==============================================================

I/O speed : 141 MB/s
[/bash]

4. Other useful commands for checking Linux server:

Check memory usage:[bash]free -m[/bash]
Check disk usage:[bash]df -h[/bash]
Check inodes usage:[bash]df -i[/bash]
Find most consumed inodes folders:[bash]find / -xdev -printf ‘%h\n’ | sort | uniq -c | sort -k 1 -n[/bash]
Check CPU cores and speed:[bash]cat /proc/cpuinfo[/bash]
Check top resources usage process:[bash]top[/bash]
List top 10 RAM/CPU usage processes:[bash]ps axo size,pcpu,command,pid | sort -rn | head -n 10[/bash]
. Change the order of arguments of ps axo so that it can be sorted by the following sort command.
Check how old is your HDD:[bash]smartctl -a /dev/sda | grep Power_On[/bash]
. Or if you are in HW RAID1, you can use
[bash]smartctl -a -d cciss,0 /dev/sda | grep Power_On[/bash]
, or can use smartctl -a -c /dev/sg2. Of course you will need to have smartctl installed (in debian:
[bash]apt-get install -y smartmontools[/bash]
or
[bash]aptitude -t wheezy-backports install -y smartmontools[/bash]
if you need the last version.).
To check health of NVMe disk, install nvme-cli package and run
```
# nvme list
# nvme smart-log /dev/nvme0
```
To check bad sectors in HDD:[bash]badblocks -o sdb.txt -vs /dev/sdb[/bash]
. Replace /dev/sdb with /dev/sda or other devices to check other HDD. Further test with smartctl can be found HERE.
To suppress all messages from the kernel (and its drivers) except panic messages from appearing on the console:
```
dmesg -n 1
```
Remember to put it to “/etc/rc.local” so that it will affect in each reboot.
```
echo "dmesg -n 1" >> /etc/rc.local
echo "" >> /etc/rc.local
```

To remount “/tmp/” and prevent it from executing files (malware):

mount -o bind,noexec /tmp /tmp
umount /tmp
mount -o bind /tmp /tmp
mount -o remount,noexec /tmp
rm -fr /tmp/*
rm -fr /var/tmp/*

To find a full path of a specific process:
```
file /proc/PROCESS_ID/exe
```
To remove all history command lines:[bash]cat /dev/null > ~/.bash_history && history -c && exit[/bash]
If a partition is not mounted with full capacity of the whole disk, we can easily extend it with[bash]resize2fs /dev/sda[/bash]

To watch number of active connections of apache processes (w/ available memory): [bash]watch -n 1 "echo -n 'Apache Processes: ' && ps -C httpd --no-headers | wc -l && free -m"[/bash]