A part of my bachelor thesis deals with benchmarking. I barely found any useful information so here is my write-up to spare yourself the trouble ...
WIPS - Whetstone Instructions Per Second
The Whetstone-Benchmark measures it's results in this unit.
DIPS - Dhrystone Instructions Per Second
In contrast to WIPS this result of the Dhrystone-Benchmark does not offer significantly different results, whether the underlying machine is RISC or CISC. The results are therefor a little more meaningful, although questionable on modern machines. 1 DMIPS is equal to 1757 DIPS.
FLOPS - Floating-Point Operiations Per Second
As this is hard to calculate or test in general, FLOPS are often derived from the benchmarks own rating system.
IPS - Instructions Per Second
IPS are the Integer equivalent of FLOPS and represent the Integer-Instructions a CPU can do in a single second.
This benchmark roots back to 1972 and measures the performance in WIPS. It mainly uses floating-point arithmetics (add or multiply floats) to measure the performance. A detailed writeup on the whetstone benchmark can be found here.
About 10 years after the creation of Whetstone a new benchmark was created. Dhrystone, in contrast to Whetstone, does not contain Floating-Point instructions. Being in use for 30 years the benchmark struggles with modern techniques like compiler-optimization and is mainly used for benchmarking on weaker machines. For embedded systems it offers a rather simple benchmark, as it can be made to run on almost every platform. A more detailed write up is available.
LINPACK has originally been a library to perform linear algebra operation but was quickly adopted as a benchmark. A parallelized version is used to rate supercomputers.