How to run the benchmark suite

From Eigen
Revision as of 15:08, 27 June 2014 by Chtz (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Here are some notes explaining how to configure and run the benchmark suite.

Quick start guide

# create a working directory
$ mkdir build
$ cd build
# create the makefiles
$ cmake <path_to_eigen_source> -DCMAKE_BUILD_TYPE=Release -DEIGEN_BUILD_BTL=ON -DEIGEN3_INCLUDE_DIR=<path_to_eigen_source>
# compile the benchmarks
$ cd bench/btl
$ make -j4
# run the benchmark (on a single core)
# VERY IMPORTANT : logout, log into a console (ctrl+shift+F1) and shutdown your X server (e.g.: sudo init 3), and stop as most as services as you can
$ OMP_NUM_THREADS=1 ctest -V
# - go sleep -
# copy the results into the same directory
$ mkdir data/set1
$ cp libs/*/*.dat data/set1
# build the nice plots
$ cd data
$ ./go_mean set1/
# the plots (png and pdf files) are in set1/



BTL_CONFIG is an environment variable used to control some parameters:

  • -a <string>: allow to run a single or a subset of the benchs. Example:
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix" ctest -V

This example run all benchs containing "matrix_matrix" into their name (check the name of the files in eigen/bench/btl/actions/ for the list of all bench).

  • -t <integer>: set the number of times a test will be run. Default is 3. We keep the best result. If you run a bench multiple times, then the old results are read back and only the best scores will be preserved. In other words the following two examples are equivalents:
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix -t 10" ctest -V
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix -t 4" ctest -V
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix -t 6" ctest -V
  • --overwrite: do not read back the previous results and overwrite them.
  • --nocheck: disable the verification of the computations which can be very time consuming.
  • --real: use the real time instead of the CPU time. Useful to bench with multithreading on.

Configure the tested libraries

  • Use ccmake or cmake-gui to enable/disable the compilation and testing of some libs. Check the variables starting with BUILD_btl_).


  • Depending on how ATLAS have been compiled you might need to manually add a link reference to the reference lapack lib. Using ccmake or cmake-gui, edit the ATLAS_LAPACK variable and append, e.g.: ";/usr/lib64/".
  • If ATLAS has not been installed in default paths, then you can set the ATLASDIR environment variable to help cmake find the libs. You can also set the ATLAS_* variable by hand. Here is an example of their values:
  ATLAS_CBLAS          /opt/atlas/lib/libcblas.a                                                                                                                                                                                       
  ATLAS_F77BLAS       /opt/atlas/lib/libf77blas.a                                                                                                                                                                                     
  ATLAS_INCLUDES      /usr/include                                                                                                                                                                                                    
  ATLAS_LAPACK        /opt/atlas/lib/liblapack.a;/usr/lib64/                                                                                                                                                        
  ATLAS_LIB           /opt/atlas/lib/libatlas.a
  • It is recommended to compile ATLAS for your specific architecture.


  • If you installed it using the default installation path, then it should be detected automatically. Otherwise you can set the MKLLIB environment variable to the path containing the MKL libraries matching your architecture. You can also set the MKL_LIBRARIES, MKL_INCLUDES, and MKL_GUIDE cmake variables yourself. Here is an example of their values:
  MKL_GUIDE            /opt/intel/Compiler/11.0/074/lib/intel64/                                                                                                                                                            
  MKL_INCLUDES         /usr/include                                                                                                                                                                                                    
  MKL_LIBRARIES        /opt/intel/Compiler/11.0/074/mkl/lib/em64t/


  • It is easy to download and compile.
  • By default we search for a library named gotoblas. You can manually set it via the GOTO_LIBRARIES cmake variable

Build Example

If you safe the contents below to a file, adapt the paths and make it executable, it should be able to configure eigen properly and run the tests for you.

export ACMLDIR=/opt/mw/acml4.4.0/ifort64_mp
export MKLDIR=/opt/mw/mkl-10.0
export MKLLIB=$MKLDIR/lib/em64t
export GOTODIR=/opt/mw/GotoBLAS2
#export EIGEN2DIR=/opt/mw/eigen2/
export EIGEN3DIR=/home/milianw/projects/eigen
export BUILDDIR=/opt/mw/eigen3-tests
cd bench/btl
make -j8