HP Vertica database validation Scripts
Hp Vertica comes with a set of utility validation scripts that can help you encounter performance issues before you install a Vertica database or after you install it. The location of the validation scripts is : /opt/vertica/bin We have three scripts:
- Vcpuperf - a CPU performance test used to verify your CPU performance.
- Vioperf - an Input/Output test used to verify the speed and consistency of your hard drives.
- Vnetperf - a Network test used to test the latency and throughput of your network between hosts.
Vcpuperf
The script will test the speed of your CPU, the script will provide you with a set of values that should apply as per each CPU model.- the returned values are in seconds and the lower the returned value the better is the performance of your CPU's.
/opt/vertica/bin/Vcpuperf [-q]- this utility comes with only one option and that is the -q which enable the command to run in quite mode.
[dbadmin@node1 bin]$ ./vcpuperf
Compiled with: 4.1.2 20080704 (Red Hat 4.1.2-55)
Expected time on Core 2, 2.53GHz: ~9.5s
Expected time on Nehalem, 2.67GHz: ~9.0s
Expected time on Xeon 5670, 2.93GHz: ~8.0s
This machine's time:
CPU Time: 13.600000s
Real Time:13.740000s
Some machines automatically throttle the CPU to save power.
This test can be done in <100 microseconds (60-70 on Xeon 5670, 2.93GHz).
Low load times much larger than 100-200us or much larger than the corresponding high loa
d time
indicate low-load throttling, which can adversely affect small query / concurrent perf
ormance.
This machine's high load time: 145 microseconds.
This machine's low load time: 271 microseconds.
#My CPU Model type is :
[dbadmin@node1 bin]$ cat /proc/cpuinfo | grep 'model name'
model name : Intel(R) Xeon(R) CPU E7540 @ 2.00GHz- well as HP recommends you should disable CPU frequency scaling thou this might cost you in terms of energy.
Vioperf
- next tool we have is going to measure the input and output of your Vertica nodes.
- sequential write
- sequential rewrite
- sequential read
- skip read (read non-contiguous data blocks)
vioperf [--help] [--duration=<INTERVAL] [--log-interval=<INTERVAL] [--log-file=<FILE]
[--condense-log] [<DIR*]Example :
- i chose to run the script using the option duration that the test will run, by default if you don't use any option the script will run for 5 minutes.
you also can use the DIR option where you point to a specific directory otherwise the current working directory will used.
[dbadmin@node1 ~]$ /opt/vertica/bin/vioperf --duration=60s
The minimum required I/O is 20 MB/s read and write per physical processor core on each nod
e, in full duplex i.e. reading and writing at this rate
simultaneously, concurrently on all nodes of the cluster.
The recommended I/O is 40 MB/s per physical core on each node. For example, the I/O rate f
or a
server node with 2 hyper-threaded six-core CPUs is 240 MB/s required minimum, 480 MB/s rec
ommended.
Using direct io (buffer size=1048576, alignment=512) for directory "/home/dbadmin"
test | directory | counter name | counte
r value | counter value (10 sec avg) | counter value/core | counter value/core
(10 sec avg) | thread count | %CPU | %IO Wait | elapsed time (s)| remaining time (s)
------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------
--------------------------
Write | /home/dbadmin | MB/s | 163
| 163 | 27.1667 | 27.1667
| 6 | 15 | 69 | 10 | 5
Write | /home/dbadmin | MB/s | 149
| 119 | 24.8333 | 19.8333
| 6 | 11 | 70 | 15 | 0
ReWrite | /home/dbadmin | (MB-read+MB-write)/s| 97+97
| 97+97 | 16.1667+16.1667 | 16.1667+16.1667
| 6 | 10 | 73 | 10 | 5
ReWrite | /home/dbadmin | (MB-read+MB-write)/s| 91+91
| 81+81 | 15.1667+15.1667 | 13.5+13.5 | 6 | 9 | 60 | 15 | 0
Read | /home/dbadmin | MB/s | 194
| 194 | 32.3333 | 32.3333 | 6 | 17 | 66 | 10 | 5
Read | /home/dbadmin | MB/s | 204
| 227 | 34 | 37.8333 | 6 | 27 | 61 | 15 | 0
SkipRead | /home/dbadmin | seeks/s | 13090
| 13090 | 2181.67 | 2181.67 | 6 | 10 | 52 | 10 | 5
SkipRead | /home/dbadmin | seeks/s | 15007
| 18845 | 2501.17 | 3140.83 | 6 | 21 | 57 | 15 | 0The output result can be read as follows: We will describe the columns that are outputted: test - this represents the type of test done. directory - what directory is used for the test. counter name - can be MB/s or seeks/s counter value - the value of the counter counter value average - avg value for the counter thread count - number of threads used for the test %CPU - the available CPU % during the test %IO wait - time that the process is locked waiting for I/O operation to complete elapsed time - duration of the test Note: The script will offer you some baseline values that you can use to benchmark your system. Such as minimum required I/O value and recommended I/O value per physical core on each node.
