Test Linux Disk IO Performance

As a DBA, sometime we need know how the disk IO performs  in Linux when database hits performance issue. Particularity SQLs have high IO waitings.

1)  Discard caches before testing :

# sync; echo 3 >  /proc/sys/vm/drop_caches

The sync command allows the kernel write as many dirty cache pages to disk as it can.
To clear PageCache only run: # sync; echo 1 > /proc/sys/vm/drop_caches
To clear dentries (Also called as Directory Cache) and inodes run:  # sync; echo 2 > /proc/sys/vm/drop_caches

Page cache is memory held after reading files. Linux kernel prefers to keep unused page cache assuming files being read once will most likely to be read again in the near future, hence avoiding the performance impact on disk IO.

dentry and inode_cache are memory held after reading directory/file attributes, such as open() and stat(). dentry is common across all file systems, but inode_cache is on a per-file-system basis. Linux kernel prefers to keep this information assuming it will be needed again in the near future, hence avoiding disk IO.

2) To measure server throughput (write speed) :

$dd if=/dev/zero of=/tmp/test1.dmp bs=1024k count=1024 oflag=dsync

1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 3.87519 s, 277 MB/s

3) To measure server latency ( read speed ):

$ dd of=/dev/zero if=/tmp/test1.dmp bs=1024k count=1024 oflag=dsync
  • oflag=dsync (oflag=dsync) : Use synchronised I/O for data. Do not skip this option. This option get rid of caching and gives you good and accurate results.
  • conv=fdatasyn: Again, this tells dd to require a complete “sync” once, right before it exits. This option is equivalent to oflag=dsync.

4) File checksum:

-- Md5sum

$ time /usr/bin/md5sum /tmp/test1.dmp
cd573cfaace07e7949bc0c46028904ff /tmp/test1.dmp

real 0m1.861s
user 0m1.609s
sys 0m0.243s

-- Sha1 

$ time sha1sum  /tmp/test1.dmp
2a492f15396a6768bcbca016993f4b4c8b0b5307  /tmp/test1.dmp

real    0m1.296s
user    0m1.050s
sys     0m0.241s

-- Sha256

$ time sha256sum  /tmp/test1.dmp
49bc20df15e412a64472421e13fe86ff1c5165e18b2afccf160d4dc19fe68a14 
/tmp/test1.dmp

real    0m2.520s
user    0m2.244s
sys     0m0.258s

How to Use WGET to Download Oracle Binary Software from MOS

Download Oracle software and patches in batch or command line, even in background.

SITUATION

Due to security or firewall issue, we have to download Oracle software and patches from a sever where web browser is unavailable. Here is an example how to download the patches by using wget utility.

IMPLEMENTATION

1) Search the patch you want.

patch_update_1

2) Select and click Download button.

patch_update_2

3) Click “View Digest Details

patch_update_3
patch_update_3_1

4) Click “WGET Options” and “Download.sh“,  save wget.sh.

patch_update_4

5) Edit wget.sh by specifying the variables with right values as below :

# SSO username and password
SSO_USERNAME=EXAMPLE@ORACLE.COM
SSO_PASSWORD=password
# Path to wget command
WGET="/usr/bin/wget --no-check-certificate --secure-protocol=TLSv1 "
patch_update_5

6) Run wget,sh script.

7) Double check the download files are OK by comparing the output string with the values from STEP 3.

patch_update_7

8) wget.sh script content.

#!/bin/sh -x
#
# Generated 6/14/15 10:25 AM
# Start of user configurable variables
#
LANG=C
export LANG
# SSO username and password
SSO_USERNAME=EXAMPLE@ORACLE.COM
SSO_PASSWORD=password
# Path to wget command
WGET="/usr/bin/wget --no-check-certificate --secure-protocol=TLSv1 "
# Location of cookie file
COOKIE_FILE=/tmp/$.cookies
# Log directory and file
LOGDIR=.
LOGFILE=$LOGDIR/wgetlog-`date +%m-%d-%y-%H:%M`.log
# Output directory and file
OUTPUT_DIR=.
#
# End of user configurable variable
#
if [ "$SSO_PASSWORD " = " " ]
then
 echo "Please edit script and set SSO_PASSWORD"
 exit
fi
# Contact updates site so that we can get SSO Params for logging in
SSO_RESPONSE=`$WGET --user-agent="Mozilla/5.0" https://updates.oracle.com/Orion/Services/download 2>&1|grep Location`
# Extract request parameters for SSO
SSO_TOKEN=`echo $SSO_RESPONSE| cut -d '=' -f 2|cut -d ' ' -f 1`
SSO_SERVER=`echo $SSO_RESPONSE| cut -d ' ' -f 2|cut -d 'p' -f 1,2`
SSO_AUTH_URL=sso/auth
AUTH_DATA="ssousername=$SSO_USERNAME&password=$SSO_PASSWORD&site2pstoretoken=$SSO_TOKEN"
# The following command to authenticate uses HTTPS. This will work only if the wget in the environment
# where this script will be executed was compiled with OpenSSL. Remove the --secure-protocol option
# if wget was not compiled with OpenSSL
# Depending on the preference, the other options are --secure-protocol= auto|SSLv2|SSLv3|TLSv1
$WGET --user-agent="Mozilla/5.0" --secure-protocol=TLSv1 --post-data $AUTH_DATA --save-cookies=$COOKIE_FILE --keep-session-cookies $SSO_SERVER$SSO_AUTH_URL -O sso.out >> $LOGFILE 2>&1
rm -f sso.out
$WGET --user-agent="Mozilla/5.0" --load-cookies=$COOKIE_FILE --save-cookies=$COOKIE_FILE --keep-session-cookies "https://updates.oracle.com/Orion/Services/download/p20819644_1121030_Linux-x86-64.zip?aru=18865551&patch_file=p20819644_1121030_Linux-x86-64.zip" -O $OUTPUT_DIR/p20819644_1121030_Linux-x86-64.zip >> $LOGFILE 2>&1
# Cleanup
rm -f $COOKIE_FILE