Difference between revisions of "Tests"

From Eigen
Jump to: navigation, search
(Writing unit tests)
Line 72: Line 72:
  
 
* Update your mercurial clone to the desired branch:
 
* Update your mercurial clone to the desired branch:
   cd path/to/eigen2
+
   cd path/to/eigen
 
   hg pull -u
 
   hg pull -u
  
Line 80: Line 80:
 
   mkdir cdash
 
   mkdir cdash
 
   cd cdash
 
   cd cdash
   cp path/to/eigen2/test/testsuite.cmake .
+
   cp path/to/eigen/test/testsuite.cmake .
 
* run the testsuite:
 
* run the testsuite:
 
   ctest -VV -S testsuite.cmake,EIGEN_BUILD_STRING=fedora-10-`uname -m`-gcc-4.3.2,EIGEN_CXX=g++-4.3,EIGEN_MODE=Experimental
 
   ctest -VV -S testsuite.cmake,EIGEN_BUILD_STRING=fedora-10-`uname -m`-gcc-4.3.2,EIGEN_CXX=g++-4.3,EIGEN_MODE=Experimental
Line 96: Line 96:
  
 
If you want to add a simple test of small feature highly related to an existing unit test, then simply extend the existing one. Otherwise, you have to create a new unit test. Let's call it ''mytest''.
 
If you want to add a simple test of small feature highly related to an existing unit test, then simply extend the existing one. Otherwise, you have to create a new unit test. Let's call it ''mytest''.
* Create a new file ''mytest.cpp'' in the eigen2/test (or eigen2/unsupported/test) folder.
+
* Create a new file ''mytest.cpp'' in the eigen/test (or eigen/unsupported/test) folder.
 
* Your file must start by including the file main.h and define a function test_mytest. CMake will take care of letting the main function know that it should call test_mytest(). Do as follows:
 
* Your file must start by including the file main.h and define a function test_mytest. CMake will take care of letting the main function know that it should call test_mytest(). Do as follows:
 
   #include <main.h>
 
   #include <main.h>
Line 103: Line 103:
 
     /* ... */
 
     /* ... */
 
   }
 
   }
* If you don't know how to fill the blank, have a look at the other unit test files in eigen2/test.
+
* If you don't know how to fill the blank, have a look at the other unit test files in eigen/test.
 
* Add it to the respective CMakeLists.txt file by adding the line:
 
* Add it to the respective CMakeLists.txt file by adding the line:
 
   ei_add_test(mytest)
 
   ei_add_test(mytest)

Revision as of 20:13, 25 November 2009

Prerequisites

First of all, this page applies to the development branch, not 2.0.

You need to know how to use CMake, see our page on that topic.

From now on, we assume that we are in the build directory and that you have already run cmake. There is no need to pass a particular option to CMake to enable tests.

Building and running unit tests

To build and run all tests using CTest, use the check target. For example, if your platform uses make:

 $ make check

This will not sent a report to the online dashboard. If you want to do that, see below.

We do not use the "test" target because of a well-documented limitation of CTest (the "test" target does not build the tests).

If you only want to build the tests, not run them, use the buildtests target. For example, with make, do:

 $ make buildtests

Working on one specific test

Suppose that a particular test, say "basicstuff", has failed. If you want to build specifically this test, just prepend it with "test_". For example:

 $ make test_basicstuff

If you want to run specifically that test and see the error messages it produces, you can run it directly:

 $ test/test_basicstuff

Note that these test executables take optional command-line parameters to control the number of repetitions and the seed. To get some help:

 $ test/test_basicstuff help

Filtering tests with regular expressions

We have two scripts for that, directly in your build directory. They currently assume "make" and the Bash shell, feel free to port them to your platform.

To build all tests matching a regular expression pattern, do:

 $ ./buildtests regexp [jobs]

The optional second parameter is to launch multiple build jobs ("make -j").

To build and run all tests matching a regular expression pattern, do:

 $ ./check regexp [jobs]

Notice that that is the same as doing "./buildtests" followed by "ctest -R".

Build types and debugging

There are 2 build types, Debug and Release.

The default build type is "Release".

If you have a test that fails and want to debug into it, you'll probably have to rebuild it in Debug mode. This is controlled by the standard CMake variable CMAKE_BUILD_TYPE. On Windows/MSVC, this is controlled from within the IDE as usual. Otherwise, we provide scripts for re-running CMake with that variable changed, just do:

 $ ./debug

and

 $ ./release

Note that in Debug mode, optimization is disabled and full debugging info is generated. Tests take a ton of disk space, and a very long time to finish.

Submitting test suite reports to the dashboard

To submit reports to the dashboard:

  • Update your mercurial clone to the desired branch:
 cd path/to/eigen
 hg pull -u

Make sure you are in the branch that you want (hg branch, hg up <branchname>).

  • Create a folder for the tests and copy the script:
 mkdir cdash
 cd cdash
 cp path/to/eigen/test/testsuite.cmake .
  • run the testsuite:
 ctest -VV -S testsuite.cmake,EIGEN_BUILD_STRING=fedora-10-`uname -m`-gcc-4.3.2,EIGEN_CXX=g++-4.3,EIGEN_MODE=Experimental

Of course, change the value of the variable to match your system. The full list of options with description is available in the testsuite.cmake file.

Then you can check the results on our dashboard: http://eigen.tuxfamily.org/CDash.

For further information, check theses web pages:

Writing unit tests

If you want to add a simple test of small feature highly related to an existing unit test, then simply extend the existing one. Otherwise, you have to create a new unit test. Let's call it mytest.

  • Create a new file mytest.cpp in the eigen/test (or eigen/unsupported/test) folder.
  • Your file must start by including the file main.h and define a function test_mytest. CMake will take care of letting the main function know that it should call test_mytest(). Do as follows:
  #include <main.h>
  void test_mytest()
  {
    /* ... */
  }
  • If you don't know how to fill the blank, have a look at the other unit test files in eigen/test.
  • Add it to the respective CMakeLists.txt file by adding the line:
  ei_add_test(mytest)

Advanced: optional ei_add_test parameters

The macro ei_add_test takes two optional arguments:

  • The first one specifies additional compile flags.
  • The second one contains the additional libraries to link with.

Advanced: splitting your test into small executables

To limit compiler memory usage and enable more parallel builds, it's good to split tests into smaller executables. It makes the most sense to group together instantiations of the same types, e.g. one executable with MatrixXf, one with Matrix4d, etc... Fortunately, we have infrastructure that makes doing that very easy.

In your test file, just replace CALL_SUBTEST by CALL_SUBTEST_1, CALL_SUBTEST_2, etc... (can go up to 16) and CMake will automatically parse that, create multiple executables, each with only one channel activated. For code that doesn't go through CALL_SUBTEST, you can directly check for this symbol: EIGEN_TEST_PART_i where i=1,2,... and again, CMake parses for that and creates corresponding executables.

See existing test files for examples.