Difference between revisions of "How to run the benchmark suite"

From Eigen
Jump to: navigation, search
(Quick start guide)
Line 1: Line 1:
 +
[[Category:Developer]]
 +
 
Here are some notes explaining how to configure and run the benchmark suite.
 
Here are some notes explaining how to configure and run the benchmark suite.
  

Revision as of 09:11, 28 January 2011


Here are some notes explaining how to configure and run the benchmark suite.

Quick start guide

# create a working directory
$ mkdir build
$ cd build
# create the makefiles
$ cmake <path_to_eigen_source> -DCMAKE_BUILD_TYPE=Release -DEIGEN_BUILD_BTL=ON -DEIGEN3_INCLUDE_DIR=<path_to_eigen_source>
# compile the benchmarks
$ cd bench/btl
$ make -j4
# run the benchmark (on a single core)
# VERY IMPORTANT : logout, log into a console (ctrl+shift+F1) and shutdown your X server (e.g.: sudo init 3), and stop as most as services as you can
$ OMP_NUM_THREADS=1 ctest -V
# - go sleep -
# copy the results into the same directory
$ mkdir data/set1
$ cp libs/*/*.dat
# build the nice plots
$ cd data
$ ./go_mean set1/
# the plots (png and pdf files) are in set1/

Tips

BTL_CONFIG

BTL_CONFIG is an environment variable used to control some parameters:

  • -a <string>: allow to run a single or a subset of the benchs. Example:
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix" ctest -V

This example run all benchs containing "matrix_matrix" into their name (check the name of the files in eigen/bench/btl/actions/ for the list of all bench).

  • -t <integer>: set the number of times a test will be run. Default is 3. We keep the best result. If you run a bench multiple times, then the old results are read back and only the best scores will be preserved. In other words the following two examples are equivalents:
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix -t 10" ctest -V
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix -t 4" ctest -V
  $ OMP_NUM_THREADS=1 BTL_CONFIG="-a matrix_matrix -t 6" ctest -V
  • --overwrite: do not read back the previous results and overwrite them.
  • --nocheck: disable the verification of the computations which can be very time consuming.
  • --real: use the real time instead of the CPU time. Useful to bench with multithreading on.

Configure the tested libraries

  • Use ccmake or cmake-gui to enable/disable the compilation and testing of some libs. Check the variables starting with BUILD_btl_).

ATLAS

  • Depending on how ATLAS have been compiled you might need to manually add a link reference to the reference lapack lib. Using ccmake or cmake-gui, edit the ATLAS_LAPACK variable and append, e.g.: ";/usr/lib64/liblapack.so.3.2.1".
  • If ATLAS has not been installed in default paths, then you can set the ATLASDIR environment variable to help cmake find the libs. You can also set the ATLAS_* variable by hand. Here is an example of their values:
  ATLAS_CBLAS          /opt/atlas/lib/libcblas.a                                                                                                                                                                                       
  ATLAS_F77BLAS       /opt/atlas/lib/libf77blas.a                                                                                                                                                                                     
  ATLAS_INCLUDES      /usr/include                                                                                                                                                                                                    
  ATLAS_LAPACK        /opt/atlas/lib/liblapack.a;/usr/lib64/liblapack.so.3.2.1                                                                                                                                                        
  ATLAS_LIB           /opt/atlas/lib/libatlas.a
  • It is recommended to compile ATLAS for your specific architecture.

MKL

  • If you installed it using the default installation path, then it should be detected automatically. Otherwise you can set the MKLLIB environment variable to the path containing the MKL libraries matching your architecture. You can also set the MKL_LIBRARIES, MKL_INCLUDES, and MKL_GUIDE cmake variables yourself. Here is an example of their values:
  MKL_GUIDE            /opt/intel/Compiler/11.0/074/lib/intel64/libguide.so                                                                                                                                                            
  MKL_INCLUDES         /usr/include                                                                                                                                                                                                    
  MKL_LIBRARIES        /opt/intel/Compiler/11.0/074/mkl/lib/em64t/libmkl_core.so

GOTO-BLAS

  • It is easy to download and compile.
  • By default we search for a library named gotoblas. You can manually set it via the GOTO_LIBRARIES cmake variable