Difference between revisions of "Developer's Corner"

From Eigen
Jump to: navigation, search
m (Debugging under Visual Studio: grammar fixes)
(Mercurial workflow)
Line 86: Line 86:
 
Notes explaining aspects of the architecture of Eigen 3.x are available at the [[Eigen3 Developer Documentation]] page
 
Notes explaining aspects of the architecture of Eigen 3.x are available at the [[Eigen3 Developer Documentation]] page
  
= Mercurial workflow =
+
= Git workflow =
  
The contents of this section have moved to [[Mercurial]].
+
TODO: you can help writing this section taking inspiration from our deprecated [[Mercurial]] page.
 +
 
 +
= Make a new release =
 +
 
 +
These days it's very easy, as the docs are automatically generated and uploaded, and the tarballs are auto-generated by GitLab.
 +
 
 +
So here's what it takes to make a new release (a new minor version):
 +
* Announce a week in advance on the [[Main_Page#Mailing_list|eigen mailing list]] the upcoming release (give a precise date) so that people can test and/or speak up if they know of an issue. Of course, in case of an emergency, no need to do that :)
 +
* Concert with other devs, and/or use CDash, and/or test yourself, to make sure that the test suite passes at the very least for:
 +
** latest stable GCC
 +
** some older GCC 4.x (ok, it's hard to check them all), try to cover the oldest GCC we're supposed to support (see main page)
 +
** latest MSVC
 +
** latest Clang
 +
* Also do think to check "make install"! Remember how it spoiled the 2.0.7 release!
 +
** Supreme refinement would be to check building the test suite against the installed Eigen.
 +
* Update the version number (EIGEN_MINOR_VERSION, etc) in the file Eigen/src/Core/util/Macros.h.
 +
* commit that:
 +
$ hg commit
 +
* Make a tag:
 +
$ hg tag 4.5.6-beta7
 +
* recommended: check that everything is in order.
 +
$ hg view
 +
* push the release:
 +
$ hg push
 +
* For a major version, write and keep up-to-date a dedicated page on the wiki, e.g. see: http://eigen.tuxfamily.org/index.php?title=3.4
 +
* For a minor version: make the full changelog. It's not just a dump of the commit messages: it must be readable and interesting for Eigen users. You can use "hg view" again, try to put the grave bug fixes first, try to group together what's related. You can also use hg and sed to generate an initial raw dump of the changesets:
 +
  hg log -b 3.3 -r tip:3.3.4  \
 +
  | grep -e '^changeset' -e '^summary' \
 +
  | sed '$!N;s/\n/ /' \
 +
  | sed 's/changeset:.*:\(.*\) summary:[ ]*\(.*\)/* [https:\/\/bitbucket.org\/eigen\/eigen\/commits\/\1 Changeset \1]: \2/' \
 +
  | sed 's/\([Bb]ug\) \([0-9]*\)/[http:\/\/eigen.tuxfamily.org\/bz\/show_bug.cgi?id=\2 \1 \2]/g' 
 +
* Update the [[Main_Page|Main Page]] on the wiki, that is the announcement at the top and the [[Main_Page#Download|Download]] section. Make sure that the tarball links point to the new release, and that any example of how to get the latest tag also refers to the new tag.
 +
** To add a news, simply create a new page in the "News" namespace:
 +
# enter <nowiki>"News:Title of my great announcement!"</nowiki> in the search box,
 +
# press search, and click "create a new page",
 +
# write your news as usual, preview, save... ([http://eigen.tuxfamily.org/index.php?title=Special%3AAllPages&from=&to=&namespace=100 examples])
 +
# you might have to enforce the regeneration of the mainpage by editing/saving it (because of mediawiki's cache)
 +
* Add the release notes into the [[ChangeLog]] page.
 +
* Write a mail to the [[Main_Page#Mailing_list|eigen mailing list]], paste the changelog into it.
 +
* Update the #eigen channel topic on IRC.
  
 
= Debugging tips =
 
= Debugging tips =

Revision as of 10:08, 24 July 2020


Communication between developers

Bugzilla, the mailing list and IRC (#eigen on freenode) are where live discussions happen.

We are transitioning to using Bugzilla for as many things as possible: for feature development, for tracking future Eigen versions and for TODO items. However, the mailing list remains a good communication channel for general discussion.

Contributors crediting and copyright

Everyone who made even the smallest contibution should add himself/herself to the Credits.

Everyone keeps copyright on his contributions. There is no copyright assignment.

When making a contribution on behalf of your employer, please try making it in your own name, not in your employer's name, if possible.

When you make large improvements to a source file, it is a good idea to add yourself a copyright line in it, although this is by no means necessary: you do implictly get copyright on the code you wrote.

Every commit should be made using the author's identity. We explain on the Mercurial page how to generate patches in such a way that you get credited when we import them.

Copying Permissively-Licensed Code into an MPL 2 File: Guidelines for Developers.

Code Quality

Using CMake

We have a little CMake tutorial here.

Unit tests and dashboard

Everything about unit tests and the dashboard, is on the Tests page.

Eigen hacking

Coding rules

We are not very strict about coding rules, however the trends are:

  • indentation with 2 spaces (no tabs !)
  • the best is to show a typical example of a public class:
 template<typename MatrixType
 class NameOfTheClass
 {
     typedef typename MatrixType::Scalar Scalar;
   public:
     // public functions start with lower case:
     Scalar nameOfTheFunction(int i) const;
     // public static functions start with upper case:
     static NameOfTheClass* Create(const MatrixType& mat);
   protected:
     // member attributes start with m_:
     Scalar m_attributeName;
     MatrixType m_matrix;
 };
 
 template<typename MatrixType> typename MatrixType::Scalar
 NameOfTheClass<MatrixType>::nameOfTheFunction(int i) const
 {
   Scalar res = 0;
   if(i>0)
   {
     for(int j=0; i<m_matrix.cols(); ++j,++i)
       res = ei_hypot(res, m_matrix.coeff(i,j));
   }
   else
   {
     j = 0;
     while (i<m_matrix.rows())
     {
       res = ei_hypot(res, m_matrix.coeff(i,j));
       ++i;
       ++j;
     }
   }
   return res;
 }
  • global functions and class/struct reserved for internal use start with ei_, eg:
 struct ei_assign_selector;
  • favor explicit names rather than short ones

Developer Documentation

Notes explaining aspects of the architecture of Eigen 3.x are available at the Eigen3 Developer Documentation page

Git workflow

TODO: you can help writing this section taking inspiration from our deprecated Mercurial page.

Make a new release

These days it's very easy, as the docs are automatically generated and uploaded, and the tarballs are auto-generated by GitLab.

So here's what it takes to make a new release (a new minor version):

  • Announce a week in advance on the eigen mailing list the upcoming release (give a precise date) so that people can test and/or speak up if they know of an issue. Of course, in case of an emergency, no need to do that :)
  • Concert with other devs, and/or use CDash, and/or test yourself, to make sure that the test suite passes at the very least for:
    • latest stable GCC
    • some older GCC 4.x (ok, it's hard to check them all), try to cover the oldest GCC we're supposed to support (see main page)
    • latest MSVC
    • latest Clang
  • Also do think to check "make install"! Remember how it spoiled the 2.0.7 release!
    • Supreme refinement would be to check building the test suite against the installed Eigen.
  • Update the version number (EIGEN_MINOR_VERSION, etc) in the file Eigen/src/Core/util/Macros.h.
  • commit that:
$ hg commit
  • Make a tag:
$ hg tag 4.5.6-beta7
  • recommended: check that everything is in order.
$ hg view
  • push the release:
$ hg push
  • For a major version, write and keep up-to-date a dedicated page on the wiki, e.g. see: http://eigen.tuxfamily.org/index.php?title=3.4
  • For a minor version: make the full changelog. It's not just a dump of the commit messages: it must be readable and interesting for Eigen users. You can use "hg view" again, try to put the grave bug fixes first, try to group together what's related. You can also use hg and sed to generate an initial raw dump of the changesets:
 hg log -b 3.3 -r tip:3.3.4  \
  | grep -e '^changeset' -e '^summary' \
  | sed '$!N;s/\n/ /' \
  | sed 's/changeset:.*:\(.*\) summary:[ ]*\(.*\)/* [https:\/\/bitbucket.org\/eigen\/eigen\/commits\/\1 Changeset \1]: \2/' \
  | sed 's/\([Bb]ug\) \([0-9]*\)/[http:\/\/eigen.tuxfamily.org\/bz\/show_bug.cgi?id=\2 \1 \2]/g'  
  • Update the Main Page on the wiki, that is the announcement at the top and the Download section. Make sure that the tarball links point to the new release, and that any example of how to get the latest tag also refers to the new tag.
    • To add a news, simply create a new page in the "News" namespace:
  1. enter "News:Title of my great announcement!" in the search box,
  2. press search, and click "create a new page",
  3. write your news as usual, preview, save... (examples)
  4. you might have to enforce the regeneration of the mainpage by editing/saving it (because of mediawiki's cache)
  • Add the release notes into the ChangeLog page.
  • Write a mail to the eigen mailing list, paste the changelog into it.
  • Update the #eigen channel topic on IRC.

Debugging tips

Debugging under Visual Studio

"Examplary view of the enhanced debugger output."

The Visual Studio IDE supports specialized debug visualizers for custom C/C++ types.

Eigen debug visualizers for Visual Studio 2012 and newer can be downloaded from eigen.natvis. The file is typically copied to %VSINSTALLDIR%\Common7\Packages\Debugger\Visualizers, where %VSINSTALLDIR% is the Visual Studio installation folder. More info can be found in Create Custom Views of Native Objects on MSDN.

For Visual Studio 2010 and older debug visualizers are defined in %VSINSTALLDIR%\..\Packages\Debugger\autoexp.dat. Adding the data found in this file right after the [Visualizer] section activates new visualization rules for some Eigen types (for Eigen2, use this file).

Studying assembly output

Using GCC

We have a macro EIGEN_ASM_COMMENT (currently defined only on GCC! please port it, it is in Macros.h):

You can use assembly comments to make it much easier to find the asm code corresponding to a particular portion of your code:

EIGEN_ASM_COMMENT("begin");
Vector4f u = v + 3*w;
EIGEN_ASM_COMMENT("end");

Then tell your compiler to output assembly code, for example with GCC it is the -S option, so you'd do:

 g++ myprogram.cpp -O2 -S -o myprogram.s

Very important: GCC doesn't seem to optimize correctly code inside the main() function. So make sure to put your critical code in another, non-inlined, function. You can use EIGEN_DONT_INLINE to prevent inlining of a function. But actually, the easiest might be to put your code in a function foo() taking references to the Eigen objects as arguments and not putting any main() function, and then compiling with "g++ -c". So your code could look like:

#include<Eigen/Core>
using namespace Eigen;
void foo(Vector4f& u, Vector4f& v, Vector4f& w)
{
  EIGEN_ASM_COMMENT("begin");
  u = v + 3*w;
  EIGEN_ASM_COMMENT("end");
}

And you then compile it with:

 g++ myprogram.cpp -O2 -c -S -o myprogram.s

The resulting asm code is now very easy to find, just search for "begin":

 #APP
 # 5 "x.cpp" 1
 	#begin
 # 0 "" 2
 #NO_APP
 	movss	.LC2(%rip), %xmm0
 	shufps	$0, %xmm0, %xmm0
 	mulps	(%rdx), %xmm0
 	addps	(%rsi), %xmm0
 	movaps	%xmm0, (%rdi)
 #APP
 # 7 "x.cpp" 1
 	#end
 # 0 "" 2
 #NO_APP

Using Visual Studio

In a first step you need to compile your program with the correct compiler and linker switches as given below:

 * C/C++ -> General -> Debug Information Format -> Program Database /Zi
 * C/C++ -> General -> Optimization -> Optimization -> Maximize Speed /O2
 * C/C++ -> General -> Code Generation -> Basic Runtime Checks -> Default
 * C/C++ -> General -> Code Generation -> Enable Enhanced Instruction Set -> Streaming SIMD Extensions 2 /arch:SSE2
 * Linker -> Debugging -> Generate Debug Info -> Yes /DEBUG

Optional settings are:

 * C/C++ -> Output Files -> Assembler Output -> Assembly With Source Code /FAs
 * Linker -> Advanced -> Randomized Base Address -> Disable Image Randomization /DYNAMICBASE:NO
 * Linker -> Advanced -> Fixed Base Address -> Generate a relocation section /FIXED:NO

The second last entry (Radmonized Base Address) is useful when you actually need to debug assembly code since it guarantees that each time you start your program it will occupy the same address space. The very last option (Fixed Base Address) is essential when you are working with Intel's VTune for performance profiling.

The remaining part is to correctly setup your test-bed in order to prevent the rather intelligent compiler from converting your code into NOP's when it reallized the code is unused. One way is to return the result of your computations - you won't see the assembly of the assignment since we are explicitly preventing inlining of the code via the macro EIGEN_DONT_INLINE. Using this macro is actually one of the more important aspects in order to encapsulate the code you are interested in, since we are lacking GCC's comment capabilities. A sample test-bed could look as follows:

 template <typename VectorType>
 struct VectorAddition
 {
   typedef VectorType ReturnType;
   EIGEN_DONT_INLINE static VectorType run(int)
   {
     VectorType a,b,c,d;
     return a+b+c+d;
   }
 };
 int main()
 {
   Vector4f res = VectorAddition<Vector4f>::run();
 }

Now, when you have correctly configured your project, you should be able to place a break point within the run method. When running your program by hitting F5 (Start Debugging) you should end up at your breakpoint in the run method. The final step is to hit ALT-8 and you're welcome in the wonderful world of assembly code.

Profiling tips

Using timers

In the file bench/BenchTimer.h we have a little timer that can be used for that.

Documentation tips

We use Doxygen, see the commands. See existing source code for example usage. Here are some best practices:

  • Make generous use of "see also" tags: \sa
  • Always use tags for parameters and return values (e.g. \param and \returns), that makes things easier to find in a glance.
  • If possible, add a code snippet. All you have is to add it in doc/snippets (see existing files there). CMake will complete your snippet into working form, compile it, run it, and record its output into a .out file. Then in your doxygen comment, just put:
 \include mysnippet.cpp
 Output: \verbinclude mysnippet.out
  • Alternatively, if you want to add a full self-compilable example (so you dont want CMake to add stuff to make it compilable), put your example file in doc/examples.

Resources

Linear algebra

Matrix products

CPU level optimizations

Meetings

On separate Meetings page.