/ itsummaries

Benchmarking

A part of my bachelor thesis deals with benchmarking. I barely found any useful information so here is my write-up to spare yourself the trouble ...

Guiding Values
WIPS - Whetstone Instructions Per Second

The Whetstone-Benchmark measures it's results in this unit.

DIPS - Dhrystone Instructions Per Second

In contrast to WIPS this result of the Dhrystone-Benchmark does not offer significantly different results, whether the underlying machine is RISC or CISC. The results are therefor a little more meaningful, although questionable on modern machines. 1 DMIPS is equal to 1757 DIPS.

FLOPS - Floating-Point Operiations Per Second

As this is hard to calculate or test in general, FLOPS are often derived from the benchmarks own rating system.

IPS - Instructions Per Second

IPS are the Integer equivalent of FLOPS and represent the Integer-Instructions a CPU can do in a single second.

Classic Benchmarks
Whetstone

This benchmark roots back to 1972 and measures the performance in WIPS. It mainly uses floating-point arithmetics (add or multiply floats) to measure the performance. A detailed writeup on the whetstone benchmark can be found here.

Dhrystone

About 10 years after the creation of Whetstone a new benchmark was created. Dhrystone, in contrast to Whetstone, does not contain Floating-Point instructions. Being in use for 30 years the benchmark struggles with modern techniques like compiler-optimization and is mainly used for benchmarking on weaker machines. For embedded systems it offers a rather simple benchmark, as it can be made to run on almost every platform. A more detailed write up is available.

LINPACK

LINPACK has originally been a library to perform linear algebra operation but was quickly adopted as a benchmark. A parallelized version is used to rate supercomputers.