Difference between revisions of "Developer's Corner"

From Eigen
Jump to: navigation, search
(Using GCC)
(Make a new release)
(43 intermediate revisions by 7 users not shown)
Line 1: Line 1:
 +
[[Category:Developer]]
 +
 
= Communication between developers =
 
= Communication between developers =
  
The [[Main_Page#Mailing_list|eigen mailing list]] and IRC (#eigen on freenode) are where live discussions happen.
+
[[Bugzilla]], the [[Main_Page#Mailing_list|mailing list]] and IRC (#eigen on freenode) are where live discussions happen.
 +
 
 +
We are transitioning to using Bugzilla for as many things as possible: for feature development, for tracking future Eigen versions and for TODO items. However, the mailing list remains a good communication channel for general discussion.
 +
 
 +
= Contributors crediting and copyright =
 +
 
 +
Everyone who made even the smallest contibution should add himself/herself to the [[Main Page#Credits|Credits]].
 +
 
 +
Everyone keeps copyright on his contributions. There is no copyright assignment.
 +
 
 +
When making a contribution on behalf of your employer, please try making it in your own name, not in your employer's name, if possible.
 +
 
 +
When you make large improvements to a source file, it is a good idea to add yourself a copyright line in it, although this is by no means necessary: you do implictly get copyright on the code you wrote.
  
For reference, there are various places in this wiki where other developers expect you to update information about what you are doing.
+
Every commit should be made using the author's identity. We explain on the [[Mercurial]] page how to generate patches in such a way that you get credited when we import them.
  
* [[Todo]] : general TODO
+
Copying Permissively-Licensed Code into an MPL 2 File: [http://www.mozilla.org/MPL/2.0/permissive-code-into-mpl.html Guidelines for Developers].
* [[Todo for 3.0]] : our roadmap for 3.0, it only contains the stuff remaining to be done.
+
* [[Status of unsupported modules]] : This page gives the status of the modules currently under unsupported/.
+
  
 
= Code Quality =
 
= Code Quality =
  
Nightly build reports:eigen
+
* '''Dashboards''' for unit tests with nightly build reports:
* '''Dashboards''':
+
** For the devel branch: http://manao.inria.fr/CDash/index.php?project=Eigen&display=project
** For the devel branch: http://eigen.tuxfamily.org/CDash/index.php?project=Eigen
+
** For the stable 3.3 branch: http://manao.inria.fr/CDash/index.php?project=Eigen3.3
** For the stable 2.0 branch: http://eigen.tuxfamily.org/CDash/index.php?project=Eigen+2.0
+
* '''[[Performance monitoring]]'''
* '''Ohloh''' statistics: http://www.ohloh.net/p/5393
+
* '''Openhub''' statistics: https://www.openhub.net/p/eigen
  
 
= Using CMake =
 
= Using CMake =
Line 71: Line 83:
 
* favor explicit names rather than short ones
 
* favor explicit names rather than short ones
  
= Mercurial workflow =
+
== Developer Documentation ==
 +
Notes explaining aspects of the architecture of Eigen 3.x are available at the [[Eigen3 Developer Documentation]] page
  
Before starting, let's mention these links:
+
= Git workflow =
  http://www.selenic.com/mercurial/wiki/UnderstandingMercurial
+
  http://www.selenic.com/mercurial/wiki/Tutorial
+
  
== Basic usage ==
+
TODO: you can help writing this section taking inspiration from our deprecated [[Mercurial]] page.
  
===Configuring Mercurial===
+
= Make a new release =
  
Before you start comitting to the Eigen respository, it's important that you have configured your identity, so that you will be credited in the hg history. To do so, edit your mercurial config file. The global config file is ~/.hgrc (in your home directory), and the clone specific config file is .hg/hgrc inside the clone directory. We recommend to use the global config file unless you really know what you are doing. You need to add a line like this:
+
These days it's very easy, as the docs are automatically generated and uploaded, and the tarballs are auto-generated by GitLab.
 
+
  [ui]
+
  username = John Smith <johnny@something.com>
+
 
+
Note that it is '''very important''' to follow this convention (RealName <email>). First, we want to know the real names and e-mail of people who contribute to our project; second, our scripts, such as the one that generates the credits section on the main page, assume this convention.
+
 
+
On Windows systems, the global config file is located in your home directory (%USERPROFILE%) and is called mercurial.ini.
+
 
+
In particular for Windows users, it is furthermore important to follow the instructions given here (http://mercurial.selenic.com/wiki/Win32TextExtension) in order to prevent comitting files with Windows line endings to the repository. In the end you should add something like
+
 
+
  [extensions]
+
  hgext.win32text=
+
  [encode]
+
  ** = cleverencode:
+
  [decode]
+
  ** = cleverdecode:
+
  [patch]
+
  eol = crlf
+
  [hooks]
+
  pretxncommit.crlf = python:hgext.win32text.forbidcrlf
+
 
+
to your config file.
+
 
+
===Getting the source===
+
 
+
Initial checkout:
+
 
+
  $ hg clone http://bitbucket.org/eigen/eigen
+
 
+
or if you have a write access to the repository you have two options:
+
 
+
  $ hg clone ssh://hg@bitbucket.org/eigen/eigen/        # See [http://bitbucket.org/help/using-ssh/] to setup ssh
+
  $ hg clone https://login@bitbucket.org/eigen/eigen/  # password login
+
 
+
 
+
After that, to update your checkout (get the new changes made to the repository):
+
 
+
  $ hg in            # check what's coming in
+
  $ hg pull -u      # hg pull brings the changes into your store, and -u updates
+
                    # your working directory to the latest revision.
+
                    # 'hg pull -u' is the same as 'hg pull && hg update'.
+
 
+
===Branches===
+
 
+
By default, what you now see is the development branch, called the 'default branch'. This is what was called 'trunk' in SVN.
+
 
+
  $ hg branch        # show the name the current branch
+
  $ hg branches      # show the list of branches
+
  $ hg up 2.0        # switch to 2.0 branch
+
 
+
===Local commit===
+
 
+
First of all, like with SVN, make sure you have put any new files under revision control:
+
  $ hg add <filename> # Put a file under revision control
+
 
+
Then you can examine before committing:
+
  $ hg status        # see what files are affected (same as 'svn status')
+
  $ hg diff          # show differences that are to be committed (same as 'svn diff')
+
 
+
To proceed with the local commit:
+
  $ hg commit        # take care, this will commit changes for the whole repository
+
                    # even if you are in a sub-directory (this is '''not''' as SVN does)
+
  $ hg commit .      # if you want to commit only the local directory
+
 
+
===Undoing a local commit===
+
In case you want to undo a local commit, e.g. to deal with typos in the commit log or similar, do:
+
  $ hg rollback      # roll back the last transaction
+
 
+
I suggest to '''only''' use this technique for local commits. Otherwise, it makes no sense, since it cannot be guaranteed that other users already pulled in public changes.
+
 
+
===Generating a patch===
+
 
+
Once you have made a local commit, you can generate a patch from it:
+
 
+
  $ hg export tip > somefile
+
 
+
Here, 'tip' means the latest revision. Try "hg help export" to see other options. This generates a file 'somefile' that you can then attach to an e-mail to the Eigen [[Main_Page#Mailing_list|eigen mailing list]]. If you have properly configured your identity as above before committing, then importing your patch will properly credit you.
+
 
+
===Pushing the local changes to the central repository===
+
 
+
Once you have made a local commit, you can push it to the central repository.
+
  $ hg out          # to check what you are actually pushing
+
  $ hg push          # no need to give the repository if this is the one cloned
+
You may see this error message:
+
  abort: push creates new remote heads!
+
  (did you forget to merge? use push -f to force)
+
This means that one of your modified files has been modified meanwhile in the repository. You need to merge before you can push: see next section. Do '''not''' use push -f as mentioned in that message, as there may be a conflict to resolve manually.
+
 
+
===Merge changes from another repository===
+
  $ hg in <repo>    # check what's coming in
+
  $ hg pull <repo>  # pull changesets from <repo>
+
  $ hg merge        # merge the new tip from <repo> into our working directory
+
  $ hg parents      # see the revisions that have been merged into the working directory
+
  $ hg commit        # commit the result of the merge
+
After 'hg merge', you may see this error message:
+
  warning: conflicts during merge.
+
  merging <filename> failed!
+
  0 files updated, 0 files merged, 0 files removed, x files unresolved
+
In that case, read the section "Resolving Conflicts" below.
+
 
+
===Resolving conflicts===
+
This is very similar to SVN. After a 'hg merge' has found conflicts, the files are left with special markers at the locations of conflicts, like:
+
  <<<<<<< local
+
  Hello Sir
+
  =======
+
  Hello Madam
+
  >>>>>>> other
+
You have to edit these files manually to resolve the conflict. Once a file <filename> is fixed, do:
+
  hg resolve -m <filename>  # mark the file as resolved
+
Once all the files are marked as resolved, you can proceed with committing.
+
 
+
===Viewing history===
+
The basic command here is 'hg log'. If you want to see full information about a certain revision, you can do:
+
 
+
  $ hg log --style changelog -r 1000  # examine revision 1000
+
 
+
But you can also enable a couple of extensions that provide fancier views. Edit (or create) your global .hgrc file (in your home directory) and add these lines:
+
 
+
  [extensions]
+
  hgk=
+
  hgext.graphlog =
+
 
+
You can now enjoy a graphical view of the commit history:
+
 
+
  $ hgk  # or try 'hg view' if that doesn't work
+
 
+
Or if you prefer a plain text output of the graph:
+
 
+
  $ hg glog
+
 
+
Finally, if you want something more powerful, have a look at the following external programs: hgtk and hgview. [http://bitbucket.org/tortoisehg/stable/wiki/hgtk Hgtk] allows many things, like viewing only as specific branch, viewing only merges, etc. Do:
+
 
+
  $ hgtk log
+
 
+
[http://www.logilab.org/project/name/hgview Hgview] is simpler but still allows viewing a specific branch:
+
 
+
  $ hgview
+
 
+
== Pushing many local commits as a single one ==
+
 
+
In case you love abusing of local commits and finally want to record only one, then here is the recipe:
+
  $ hg clone <central-repo> hackrepo
+
  $ cd hackrepo
+
  $ hg log | head                  # record the last rev id (=id0)
+
  # hack/commit many times
+
  $ cd ..
+
  $ hg clone <central-repo> main
+
  $ cd main
+
  $ (cd ../hackrepo ; hg export id0:) | hg import - -m "unified commit message"
+
  $ hg push
+
 
+
== Backporting bugfixes ==
+
 
+
  $ hg clone <central-repo>
+
  $ hg up 2.0              # switch your working copy to the 2.0 stable branch
+
  $ hg log | less            # find the changeset ids you want to backport
+
  $ hg transplant --log REV
+
 
+
The last command automatically commit the change using the same message than the exported commit appended with the
+
  (transplanted from REV)
+
line. If the merge failed, then you have to resolve the conflict and commit as usual. See also the [http://www.selenic.com/mercurial/wiki/TransplantExtension transplant extension] doc for more details.
+
 
+
Instead of the transplant command, you might also do:
+
  $ hg export REV | hg import -
+
or even:
+
  $ hg export REV | hg import - -m "backporting REV : short explanation"
+
 
+
== Commit review ==
+
 
+
Unless we do not find the "optimal" solution for commit reviews here is a possibility using bitbucket.org forks.
+
 
+
Assuming you have some changes in your Eigen tip you want somebody to review, you may proceed as follows.
+
 
+
First, you locally commit your changes in your cloned working copy of Eigen
+
  $ cd eigen # the cloned working copy
+
  $ hg commit -m "My meaningful commit message."
+
  $ cd ..
+
 
+
Afterwards, you need to create a fork of Eigen. Once the fork is created you can clone it
+
  $ hg clone https://login@bitbucket.org/login/eigen/ eigen_clone
+
 
+
The next step is to pull the changes from the local clone of your Eigen tip into the forked version.
+
  $ cd eigen_clone
+
  $ hg pull -u -r tip ../eigen
+
 
+
Finally, you can push back the potential commit to the Eigen fork via
+
  $ hg push
+
 
+
Your commit is now publicly visible and the developers can review it.
+
 
+
== Receiving commit notification ==
+
 
+
The Bitbucket web interface offers RSS and Atom feeds.
+
 
+
If you prefer email notification, subscribe to the eigen-commits mailing list. Instructions are the same as for the [[Main_Page#Mailing_list|eigen mailing list]], just replace "eigen" by "eigen-commits" everywhere.
+
 
+
== Search and replace on whole directory ==
+
 
+
If you do a search-and-replace operation on a whole directory, you probably do not want it to affect Mercurial's own internal files! So you must avoid all .hg directories. You can do:
+
 
+
  $ find . -type f | grep -v .hg | xargs sed -i 's/oldstring/newstring/g'
+
 
+
== Make a new release ==
+
 
+
These days it's very easy, as the docs are automatically generated and uploaded by Thomas' machine, and the tarballs are auto-generated by Bitbucket.
+
  
 
So here's what it takes to make a new release (a new minor version):
 
So here's what it takes to make a new release (a new minor version):
Line 292: Line 98:
 
* Concert with other devs, and/or use CDash, and/or test yourself, to make sure that the test suite passes at the very least for:
 
* Concert with other devs, and/or use CDash, and/or test yourself, to make sure that the test suite passes at the very least for:
 
** latest stable GCC
 
** latest stable GCC
** some older GCC 4.x (ok, it's hard to check them all)
+
** some older GCC 4.x (ok, it's hard to check them all), try to cover the oldest GCC we're supposed to support (see main page)
** oldest GCC we're supposed to support (currently 3.3)
+
 
** latest MSVC
 
** latest MSVC
 +
** latest Clang
 
* Also do think to check "make install"! Remember how it spoiled the 2.0.7 release!
 
* Also do think to check "make install"! Remember how it spoiled the 2.0.7 release!
 
** Supreme refinement would be to check building the test suite against the installed Eigen.
 
** Supreme refinement would be to check building the test suite against the installed Eigen.
 
* Update the version number (EIGEN_MINOR_VERSION, etc) in the file Eigen/src/Core/util/Macros.h.
 
* Update the version number (EIGEN_MINOR_VERSION, etc) in the file Eigen/src/Core/util/Macros.h.
* In the 2.0 branch, also update the version number in the root CMakeLists.txt. That doesn't apply to the default branch.
 
 
* commit that:
 
* commit that:
  $ hg commit
+
  git add Eigen/src/Core/util/Macros.h
 +
git commit
 
* Make a tag:
 
* Make a tag:
  $ hg tag 4.5.6-beta7
+
  git tag TAGNAME
 
* recommended: check that everything is in order.
 
* recommended: check that everything is in order.
  $ hg view
+
  git show
 
* push the release:
 
* push the release:
  $ hg push
+
  git push --tags REMOTENAME BRANCHNAME
* For a minor version: make the full changelog. It's not just a dump of the commit messages: it must be readable and interesting for Eigen users. You can use "hg view" again, try to put the grave bug fixes first, try to group together what's related.
+
e.g. to release a 3.3.X version
 +
git push --tags origin 3.3
 +
* For a major version, write and keep up-to-date a dedicated page on the wiki, e.g. see: http://eigen.tuxfamily.org/index.php?title=3.4
 +
* For a minor version: make the full changelog. It's not just a dump of the commit messages: it must be readable and interesting for Eigen users. You can use "git log" again, try to put the grafted/cherry-picked bug fixes first, try to group together what's related. You can also use git and sed to generate an initial raw dump of the changesets:
 +
  git --no-pager log --pretty=format:"* <nowiki>[https://gitlab.com/libeigen/eigen/-/commit/%H Commit %h]:</nowiki> %s" HEAD...PREVIOUSTAGNAME \
 +
    | sed 's/\([Bb]ug\) #\([0-9]*\)/[https:\/\/gitlab.com\/libeigen\/eigen\/-\/issues\/\2 Issue #\2]/g'
 +
to get the tag of the previous release you can check
 +
# Get all tags
 +
git tag
 +
# Filter for tags
 +
git tag --list 'PATTERN'
 
* Update the [[Main_Page|Main Page]] on the wiki, that is the announcement at the top and the [[Main_Page#Download|Download]] section. Make sure that the tarball links point to the new release, and that any example of how to get the latest tag also refers to the new tag.
 
* Update the [[Main_Page|Main Page]] on the wiki, that is the announcement at the top and the [[Main_Page#Download|Download]] section. Make sure that the tarball links point to the new release, and that any example of how to get the latest tag also refers to the new tag.
 +
** To add a news, simply create a new page in the "News" namespace:
 +
# enter <nowiki>"News:Title of my great announcement!"</nowiki> in the search box,
 +
# press search, and click "create a new page",
 +
# write your news as usual, preview, save... ([http://eigen.tuxfamily.org/index.php?title=Special%3AAllPages&from=&to=&namespace=100 examples])
 +
# you might have to enforce the regeneration of the mainpage by editing/saving it (because of mediawiki's cache)
 
* Add the release notes into the [[ChangeLog]] page.
 
* Add the release notes into the [[ChangeLog]] page.
 
* Write a mail to the [[Main_Page#Mailing_list|eigen mailing list]], paste the changelog into it.
 
* Write a mail to the [[Main_Page#Mailing_list|eigen mailing list]], paste the changelog into it.
Line 317: Line 138:
 
=== Debugging under Visual Studio ===
 
=== Debugging under Visual Studio ===
 
[[Image:Eigen_debug.jpg|thumb|left|"Examplary view of the enhanced debugger output."]]
 
[[Image:Eigen_debug.jpg|thumb|left|"Examplary view of the enhanced debugger output."]]
The Visual Studio IDE allows to add specialized debug visualizers for C/C++ code. The way in which specific classes are displayed in the debug output is defined by the file
+
The Visual Studio IDE supports specialized debug visualizers for custom C/C++ types.
  
    %VS90COMNTOOLS%\..\Packages\Debugger\autoexp.dat
+
Eigen debug visualizers for Visual Studio 2012 and newer can be downloaded from [https://bitbucket.org/eigen/eigen/src/tip/debug/msvc/eigen.natvis eigen.natvis]. The file is typically copied to <code>%VSINSTALLDIR%\Common7\Packages\Debugger\Visualizers</code>, where <code>%VSINSTALLDIR%</code> is the Visual Studio installation folder. More info can be found in [https://msdn.microsoft.com/en-us/library/jj620914.aspx Create Custom Views of Native Objects] on MSDN.
  
(note that the environment variable may change depending on your IDE version, e.g. VS80COMMONTOOLS).
+
For Visual Studio 2010 and older debug visualizers are defined in <code>%VSINSTALLDIR%\..\Packages\Debugger\autoexp.dat</code>. Adding the data found in [http://bitbucket.org/eigen/eigen/src/tip/debug/msvc/eigen_autoexp_part.dat this file] right after the [Visualizer] section activates new visualization rules for some Eigen types (for Eigen2, use [http://eigen.tuxfamily.org/bz/attachment.cgi?id=245 this file]).
 
+
Adding the data found in [http://eigen.tuxfamily.org/misc/eigen_autoexp_part.dat this] file right after the [Visualizer] section activates new visualization rules for some Eigen types. Currently, all kinds of Matrix types are covered wheras e.g. Eigen::SparseMatrix and Eigen::Transform are still missing. Any new visualizers are welcome and will be added with a note referencing the author to our current file.
+
 
+
Before working on the autoexp.dat file please ensure to create a backup copy!
+
  
 
=== Studying assembly output ===
 
=== Studying assembly output ===
Line 335: Line 152:
 
You can use assembly comments to make it much easier to find the asm code corresponding to a particular portion of your code:
 
You can use assembly comments to make it much easier to find the asm code corresponding to a particular portion of your code:
 
<source lang="cpp">
 
<source lang="cpp">
  EIGEN_ASM_COMMENT("begin");
+
EIGEN_ASM_COMMENT("begin");
  Vector4f u = v + 3*w;
+
Vector4f u = v + 3*w;
  EIGEN_ASM_COMMENT("end");
+
EIGEN_ASM_COMMENT("end");
 
</source>
 
</source>
 
Then tell your compiler to output assembly code, for example with GCC it is the -S option, so you'd do:
 
Then tell your compiler to output assembly code, for example with GCC it is the -S option, so you'd do:
 
   g++ myprogram.cpp -O2 -S -o myprogram.s
 
   g++ myprogram.cpp -O2 -S -o myprogram.s
  
'''Very important:''' GCC doesn't seem to optimize correctly code inside the main() function. So make sure to put your critical code in another, non-inlined, function. Actually, the easiest might be to put it in a function foo() taking references to the Eigen objects as arguments and not putting any main() function, and then compiling with "g++ -c". So your code could look like:
+
'''Very important:''' GCC doesn't seem to optimize correctly code inside the main() function. So make sure to put your critical code in another, non-inlined, function. You can use EIGEN_DONT_INLINE to prevent inlining of a function. But actually, the easiest might be to put your code in a function foo() taking references to the Eigen objects as arguments and not putting any main() function, and then compiling with "g++ -c". So your code could look like:
 
<source lang="cpp">
 
<source lang="cpp">
 
#include<Eigen/Core>
 
#include<Eigen/Core>
Line 349: Line 166:
 
{
 
{
 
   EIGEN_ASM_COMMENT("begin");
 
   EIGEN_ASM_COMMENT("begin");
   Vector4f u = v + 3*w;
+
   u = v + 3*w;
 
   EIGEN_ASM_COMMENT("end");
 
   EIGEN_ASM_COMMENT("end");
 
}
 
}
Line 426: Line 243:
 
   Output: \verbinclude mysnippet.out
 
   Output: \verbinclude mysnippet.out
 
* Alternatively, if you want to add a full self-compilable example (so you dont want CMake to add stuff to make it compilable), put your example file in doc/examples.
 
* Alternatively, if you want to add a full self-compilable example (so you dont want CMake to add stuff to make it compilable), put your example file in doc/examples.
 +
 +
= Resources =
 +
 +
== Linear algebra ==
 +
* General
 +
** [http://www.mit.edu/~wingated/stuff_i_use/matrix_cookbook.pdf The Matrix Cookbook]
 +
** [http://www.netlib.org/lapack/lawns Lapack working notes]
 +
** [http://dx.doi.org/10.1109/38.486688 Consider the lowly 2x2 matrix] (requires an IEEE subscription)
 +
* Linear iterative solvers
 +
** [http://www.netlib.org/linalg/html_templates/Templates.html Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods]
 +
** [http://www-users.cs.umn.edu/~saad/books.html Iterative methods for sparse linear systems]
 +
* Eigenvalue problem
 +
** [http://www.cs.utk.edu/~dongarra/etemplates/index.html Templates for the Solution of Algebraic Eigenvalue Problems: a Practical Guide]
 +
** [http://www-users.cs.umn.edu/~saad/books.html Numerical Methods for Large Eigenvalue Problems]
 +
** [http://www.cds.caltech.edu/~andrea/research/sw/2x2_eigen.html Eigen decomposition for a 2x2 matrix]
 +
** [http://www.geometrictools.com/Documentation/EigenSymmetric3x3.pdf Direct eigen decomposition for a 3x3 symmetric matrix]
 +
** [http://www.cs.utexas.edu/users/flame/pubs/flawn60.pdf Restructuring the QR Algorithm for High-Performance Application of Givens Rotations]
 +
 +
== Matrix products ==
 +
* [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.140.1785&rep=rep1&type=pdf Anatomy of high-performance matrix multiplication], Kazushige Goto, Robert A. van de Geijn, ACM Transactions on Mathematical Software (TOMS), Volume 34 ,  Issue 3  (May 2008)
 +
* [http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.140.2510&rep=rep1&type=pdf High-performance implementation of the level-3 BLAS] Kazushige Goto, Robert A. van de Geijn, "", ACM Transactions on Mathematical Software (TOMS), Volume 35 ,  Issue 1  (July 2008)
 +
* [http://www.alexkr.com/docs/matrixmult.pdf GotoBLAS - Anatomy of a fast matrix multiplication] Alexander Krivutsenko (A gentle introduction to GotoBLAS blocking algorithms)
 +
 +
== CPU level optimizations ==
 +
* CPUID
 +
** [http://www.sandpile.org/ia32/cpuid.htm IA-32 architecture]
 +
* [http://www.research.scea.com/gdc2003/fast-math-functions_p1.pdf fast-math-functions 1], [http://www.research.scea.com/gdc2003/fast-math-functions_p2.pdf fast-math-functions 2]
 +
 +
= Meetings =
 +
 +
On separate [[Meetings]] page.

Revision as of 16:42, 10 August 2020


Communication between developers

Bugzilla, the mailing list and IRC (#eigen on freenode) are where live discussions happen.

We are transitioning to using Bugzilla for as many things as possible: for feature development, for tracking future Eigen versions and for TODO items. However, the mailing list remains a good communication channel for general discussion.

Contributors crediting and copyright

Everyone who made even the smallest contibution should add himself/herself to the Credits.

Everyone keeps copyright on his contributions. There is no copyright assignment.

When making a contribution on behalf of your employer, please try making it in your own name, not in your employer's name, if possible.

When you make large improvements to a source file, it is a good idea to add yourself a copyright line in it, although this is by no means necessary: you do implictly get copyright on the code you wrote.

Every commit should be made using the author's identity. We explain on the Mercurial page how to generate patches in such a way that you get credited when we import them.

Copying Permissively-Licensed Code into an MPL 2 File: Guidelines for Developers.

Code Quality

Using CMake

We have a little CMake tutorial here.

Unit tests and dashboard

Everything about unit tests and the dashboard, is on the Tests page.

Eigen hacking

Coding rules

We are not very strict about coding rules, however the trends are:

  • indentation with 2 spaces (no tabs !)
  • the best is to show a typical example of a public class:
 template<typename MatrixType
 class NameOfTheClass
 {
     typedef typename MatrixType::Scalar Scalar;
   public:
     // public functions start with lower case:
     Scalar nameOfTheFunction(int i) const;
     // public static functions start with upper case:
     static NameOfTheClass* Create(const MatrixType& mat);
   protected:
     // member attributes start with m_:
     Scalar m_attributeName;
     MatrixType m_matrix;
 };
 
 template<typename MatrixType> typename MatrixType::Scalar
 NameOfTheClass<MatrixType>::nameOfTheFunction(int i) const
 {
   Scalar res = 0;
   if(i>0)
   {
     for(int j=0; i<m_matrix.cols(); ++j,++i)
       res = ei_hypot(res, m_matrix.coeff(i,j));
   }
   else
   {
     j = 0;
     while (i<m_matrix.rows())
     {
       res = ei_hypot(res, m_matrix.coeff(i,j));
       ++i;
       ++j;
     }
   }
   return res;
 }
  • global functions and class/struct reserved for internal use start with ei_, eg:
 struct ei_assign_selector;
  • favor explicit names rather than short ones

Developer Documentation

Notes explaining aspects of the architecture of Eigen 3.x are available at the Eigen3 Developer Documentation page

Git workflow

TODO: you can help writing this section taking inspiration from our deprecated Mercurial page.

Make a new release

These days it's very easy, as the docs are automatically generated and uploaded, and the tarballs are auto-generated by GitLab.

So here's what it takes to make a new release (a new minor version):

  • Announce a week in advance on the eigen mailing list the upcoming release (give a precise date) so that people can test and/or speak up if they know of an issue. Of course, in case of an emergency, no need to do that :)
  • Concert with other devs, and/or use CDash, and/or test yourself, to make sure that the test suite passes at the very least for:
    • latest stable GCC
    • some older GCC 4.x (ok, it's hard to check them all), try to cover the oldest GCC we're supposed to support (see main page)
    • latest MSVC
    • latest Clang
  • Also do think to check "make install"! Remember how it spoiled the 2.0.7 release!
    • Supreme refinement would be to check building the test suite against the installed Eigen.
  • Update the version number (EIGEN_MINOR_VERSION, etc) in the file Eigen/src/Core/util/Macros.h.
  • commit that:
git add Eigen/src/Core/util/Macros.h
git commit
  • Make a tag:
git tag TAGNAME
  • recommended: check that everything is in order.
git show
  • push the release:
git push --tags REMOTENAME BRANCHNAME

e.g. to release a 3.3.X version

git push --tags origin 3.3
  • For a major version, write and keep up-to-date a dedicated page on the wiki, e.g. see: http://eigen.tuxfamily.org/index.php?title=3.4
  • For a minor version: make the full changelog. It's not just a dump of the commit messages: it must be readable and interesting for Eigen users. You can use "git log" again, try to put the grafted/cherry-picked bug fixes first, try to group together what's related. You can also use git and sed to generate an initial raw dump of the changesets:
 git --no-pager log --pretty=format:"* [https://gitlab.com/libeigen/eigen/-/commit/%H Commit %h]: %s" HEAD...PREVIOUSTAGNAME \
   | sed 's/\([Bb]ug\) #\([0-9]*\)/[https:\/\/gitlab.com\/libeigen\/eigen\/-\/issues\/\2 Issue #\2]/g'

to get the tag of the previous release you can check

# Get all tags
git tag
# Filter for tags
git tag --list 'PATTERN'
  • Update the Main Page on the wiki, that is the announcement at the top and the Download section. Make sure that the tarball links point to the new release, and that any example of how to get the latest tag also refers to the new tag.
    • To add a news, simply create a new page in the "News" namespace:
  1. enter "News:Title of my great announcement!" in the search box,
  2. press search, and click "create a new page",
  3. write your news as usual, preview, save... (examples)
  4. you might have to enforce the regeneration of the mainpage by editing/saving it (because of mediawiki's cache)
  • Add the release notes into the ChangeLog page.
  • Write a mail to the eigen mailing list, paste the changelog into it.
  • Update the #eigen channel topic on IRC.

Debugging tips

Debugging under Visual Studio

"Examplary view of the enhanced debugger output."

The Visual Studio IDE supports specialized debug visualizers for custom C/C++ types.

Eigen debug visualizers for Visual Studio 2012 and newer can be downloaded from eigen.natvis. The file is typically copied to %VSINSTALLDIR%\Common7\Packages\Debugger\Visualizers, where %VSINSTALLDIR% is the Visual Studio installation folder. More info can be found in Create Custom Views of Native Objects on MSDN.

For Visual Studio 2010 and older debug visualizers are defined in %VSINSTALLDIR%\..\Packages\Debugger\autoexp.dat. Adding the data found in this file right after the [Visualizer] section activates new visualization rules for some Eigen types (for Eigen2, use this file).

Studying assembly output

Using GCC

We have a macro EIGEN_ASM_COMMENT (currently defined only on GCC! please port it, it is in Macros.h):

You can use assembly comments to make it much easier to find the asm code corresponding to a particular portion of your code:

EIGEN_ASM_COMMENT("begin");
Vector4f u = v + 3*w;
EIGEN_ASM_COMMENT("end");

Then tell your compiler to output assembly code, for example with GCC it is the -S option, so you'd do:

 g++ myprogram.cpp -O2 -S -o myprogram.s

Very important: GCC doesn't seem to optimize correctly code inside the main() function. So make sure to put your critical code in another, non-inlined, function. You can use EIGEN_DONT_INLINE to prevent inlining of a function. But actually, the easiest might be to put your code in a function foo() taking references to the Eigen objects as arguments and not putting any main() function, and then compiling with "g++ -c". So your code could look like:

#include<Eigen/Core>
using namespace Eigen;
void foo(Vector4f& u, Vector4f& v, Vector4f& w)
{
  EIGEN_ASM_COMMENT("begin");
  u = v + 3*w;
  EIGEN_ASM_COMMENT("end");
}

And you then compile it with:

 g++ myprogram.cpp -O2 -c -S -o myprogram.s

The resulting asm code is now very easy to find, just search for "begin":

 #APP
 # 5 "x.cpp" 1
 	#begin
 # 0 "" 2
 #NO_APP
 	movss	.LC2(%rip), %xmm0
 	shufps	$0, %xmm0, %xmm0
 	mulps	(%rdx), %xmm0
 	addps	(%rsi), %xmm0
 	movaps	%xmm0, (%rdi)
 #APP
 # 7 "x.cpp" 1
 	#end
 # 0 "" 2
 #NO_APP

Using Visual Studio

In a first step you need to compile your program with the correct compiler and linker switches as given below:

 * C/C++ -> General -> Debug Information Format -> Program Database /Zi
 * C/C++ -> General -> Optimization -> Optimization -> Maximize Speed /O2
 * C/C++ -> General -> Code Generation -> Basic Runtime Checks -> Default
 * C/C++ -> General -> Code Generation -> Enable Enhanced Instruction Set -> Streaming SIMD Extensions 2 /arch:SSE2
 * Linker -> Debugging -> Generate Debug Info -> Yes /DEBUG

Optional settings are:

 * C/C++ -> Output Files -> Assembler Output -> Assembly With Source Code /FAs
 * Linker -> Advanced -> Randomized Base Address -> Disable Image Randomization /DYNAMICBASE:NO
 * Linker -> Advanced -> Fixed Base Address -> Generate a relocation section /FIXED:NO

The second last entry (Radmonized Base Address) is useful when you actually need to debug assembly code since it guarantees that each time you start your program it will occupy the same address space. The very last option (Fixed Base Address) is essential when you are working with Intel's VTune for performance profiling.

The remaining part is to correctly setup your test-bed in order to prevent the rather intelligent compiler from converting your code into NOP's when it reallized the code is unused. One way is to return the result of your computations - you won't see the assembly of the assignment since we are explicitly preventing inlining of the code via the macro EIGEN_DONT_INLINE. Using this macro is actually one of the more important aspects in order to encapsulate the code you are interested in, since we are lacking GCC's comment capabilities. A sample test-bed could look as follows:

 template <typename VectorType>
 struct VectorAddition
 {
   typedef VectorType ReturnType;
   EIGEN_DONT_INLINE static VectorType run(int)
   {
     VectorType a,b,c,d;
     return a+b+c+d;
   }
 };
 int main()
 {
   Vector4f res = VectorAddition<Vector4f>::run();
 }

Now, when you have correctly configured your project, you should be able to place a break point within the run method. When running your program by hitting F5 (Start Debugging) you should end up at your breakpoint in the run method. The final step is to hit ALT-8 and you're welcome in the wonderful world of assembly code.

Profiling tips

Using timers

In the file bench/BenchTimer.h we have a little timer that can be used for that.

Documentation tips

We use Doxygen, see the commands. See existing source code for example usage. Here are some best practices:

  • Make generous use of "see also" tags: \sa
  • Always use tags for parameters and return values (e.g. \param and \returns), that makes things easier to find in a glance.
  • If possible, add a code snippet. All you have is to add it in doc/snippets (see existing files there). CMake will complete your snippet into working form, compile it, run it, and record its output into a .out file. Then in your doxygen comment, just put:
 \include mysnippet.cpp
 Output: \verbinclude mysnippet.out
  • Alternatively, if you want to add a full self-compilable example (so you dont want CMake to add stuff to make it compilable), put your example file in doc/examples.

Resources

Linear algebra

Matrix products

CPU level optimizations

Meetings

On separate Meetings page.