Developer's Corner

From Eigen
Revision as of 02:24, 15 November 2009 by Bjacob (Talk | contribs)

Jump to: navigation, search

Communication between developers

The eigen mailing list and IRC (#eigen on freenode) are where live discussions happen.

For reference, there are various places in this wiki where other developers expect you to update information about what you are doing.

Code Quality

Nightly build reports:

Using CMake

We have a little CMake tutorial here.

Unit tests and dashboard

Everything about unit tests and the dashboard, is on the Tests page.

Eigen hacking

Coding rules

We are not very strict about coding rules, however the trends are:

  • indentation with 2 spaces (no tabs !)
  • the best is to show a typical example of a public class:
 template<typename MatrixType
 class NameOfTheClass
 {
     typedef typename MatrixType::Scalar Scalar;
   public:
     // public functions start with lower case:
     Scalar nameOfTheFunction(int i) const;
     // public static functions start with upper case:
     static NameOfTheClass* Create(const MatrixType& mat);
   protected:
     // member attributes start with m_:
     Scalar m_attributeName;
     MatrixType m_matrix;
 };
 
 template<typename MatrixType> typename MatrixType::Scalar
 NameOfTheClass<MatrixType>::nameOfTheFunction(int i) const
 {
   Scalar res = 0;
   if(i>0)
   {
     for(int j=0; i<m_matrix.cols(); ++j,++i)
       res = ei_hypot(res, m_matrix.coeff(i,j));
   }
   else
   {
     j = 0;
     while (i<m_matrix.rows())
     {
       res = ei_hypot(res, m_matrix.coeff(i,j));
       ++i;
       ++j;
     }
   }
   return res;
 }
  • global functions and class/struct reserved for internal use start with ei_, eg:
 struct ei_assign_selector;
  • favor explicit names rather than short ones

Mercurial workflow

Before starting, let's mention these links:

 http://www.selenic.com/mercurial/wiki/UnderstandingMercurial
 http://www.selenic.com/mercurial/wiki/Tutorial

Basic usage

Configuring Mercurial

Before you start comitting to the Eigen respository, it's important that you have configured your identity, so that you will be credited in the hg history. To do so, edit your .hgrc file (either the one in your home directory, or the one in the .hg/ subdirectory of your current working directory) and add a line like this:

 [ui]
 username = John Smith <johnny@something.com>

Note that it is very important to follow this convention (RealName <email>). First, we want to know the real names of people who contribute to our project; second, our scripts, such as the one that generates the credits section on the main page, assume this convention.

On Windows systems, the config file located in your home directory (%USERPROFILE%) is called mercurial.ini and the config file located in the .hg subdirectory of your current working copy is simply called hgrc.

In particular for Windows users, it is furthermore important to follow the instructions given here (http://mercurial.selenic.com/wiki/Win32TextExtension) in order to prevent comitting files with Windows line endings to the repository. In the end you should add something like

 [extensions]
 hgext.win32text=
 [encode]
 ** = cleverencode:
 [decode]
 ** = cleverdecode:
 [patch]
 eol = crlf
 [hooks]
 pretxncommit.crlf = python:hgext.win32text.forbidcrlf

to your config file.

Getting the source

Initial checkout:

 $ hg clone http://bitbucket.org/eigen/eigen2

or if you have a write access to the repository you have two options:

 $ hg clone ssh://hg@bitbucket.org/eigen/eigen2/        # See [1] to setup ssh
 $ hg clone https://login@bitbucket.org/eigen/eigen2/   # password login


After that, to update your checkout (get the new changes made to the repository):

 $ hg in            # check what's coming in
 $ hg pull -u       # hg pull brings the changes into your store, and -u updates
                    # your working directory to the latest revision.
                    # 'hg pull -u' is the same as 'hg pull && hg update'.

Branches

By default, what you now see is the development branch, called the 'default branch'. This is what was called 'trunk' in SVN.

 $ hg branch        # show the name the current branch
 $ hg branches      # show the list of branches
 $ hg up 2.0        # switch to 2.0 branch

Local commit

First of all, like with SVN, make sure you have put any new files under revision control:

 $ hg add <filename> # Put a file under revision control

Then you can examine before committing:

 $ hg status        # see what files are affected (same as 'svn status')
 $ hg diff          # show differences that are to be committed (same as 'svn diff')

To proceed with the local commit:

 $ hg commit        # take care, this will commit changes for the whole repository
                    # even if you are in a sub-directory (this is not as SVN does)
 $ hg commit .      # if you want to commit only the local directory

Undoing a local commit

In case you want to undo a local commit, e.g. to deal with typos in the commit log or similar, do:

 $ hg rollback      # roll back the last transaction

I suggest to only use this technique for local commits. Otherwise, it makes no sense, since it cannot be guaranteed that other users already pulled in public changes.

Generating a patch

Once you have made a local commit, you can generate a patch from it:

 $ hg export tip > somefile

Here, 'tip' means the latest revision. Try "hg help export" to see other options. This generates a file 'somefile' that you can then attach to an e-mail to the Eigen eigen mailing list. If you have properly configured your identity as above before committing, then importing your patch will properly credit you.

Pushing the local changes to the central repository

Once you have made a local commit, you can push it to the central repository.

 $ hg out           # to check what you are actually pushing
 $ hg push          # no need to give the repository if this is the one cloned

You may see this error message:

 abort: push creates new remote heads!
 (did you forget to merge? use push -f to force)

This means that one of your modified files has been modified meanwhile in the repository. You need to merge before you can push: see next section. Do not use push -f as mentioned in that message, as there may be a conflict to resolve manually.

Merge changes from another repository

 $ hg in <repo>     # check what's coming in
 $ hg pull <repo>   # pull changesets from <repo>
 $ hg merge         # merge the new tip from <repo> into our working directory
 $ hg parents       # see the revisions that have been merged into the working directory
 $ hg commit        # commit the result of the merge

After 'hg merge', you may see this error message:

 warning: conflicts during merge.
 merging <filename> failed!
 0 files updated, 0 files merged, 0 files removed, x files unresolved

In that case, read the section "Resolving Conflicts" below.

Resolving conflicts

This is very similar to SVN. After a 'hg merge' has found conflicts, the files are left with special markers at the locations of conflicts, like:

 <<<<<<< local
 Hello Sir
 =======
 Hello Madam
 >>>>>>> other

You have to edit these files manually to resolve the conflict. Once a file <filename> is fixed, do:

 hg resolve -m <filename>   # mark the file as resolved

Once all the files are marked as resolved, you can proceed with committing.

Viewing history

The basic command here is 'hg log'. If you want to see full information about a certain revision, you can do:

 $ hg log --style changelog -r 1000  # examine revision 1000

But you can also enable a couple of extensions that provide fancier views. Edit (or create) your global .hgrc file (in your home directory) and add these lines:

 [extensions]
 hgk=
 hgext.graphlog =

You can now enjoy a graphical view of the commit history:

 $ hgk   # or try 'hg view' if that doesn't work

Or if you prefer a plain text output of the graph:

 $ hg glog

Finally, if you want something more powerful, have a look at the following external programs: hgtk and hgview. Hgtk allows many things, like viewing only as specific branch, viewing only merges, etc. Do:

 $ hgtk log

Hgview is simpler but still allows viewing a specific branch:

 $ hgview

Pushing many local commits as a single one

In case you love abusing of local commits and finally want to record only one, then here is the recipe:

 $ hg clone <central-repo> hackrepo
 $ cd hackrepo
 $ hg log | head                  # record the last rev id (=id0)
 # hack/commit many times
 $ cd ..
 $ hg clone <central-repo> main
 $ cd main
 $ (cd ../hackrepo ; hg export id0:) | hg import - -m "unified commit message"
 $ hg push

Backporting bugfixes

 $ hg clone <central-repo>
 $ hg up 2.0              # switch your working copy to the 2.0 stable branch
 $ hg log | less             # find the changeset ids you want to backport
 $ hg transplant --log REV

The last command automatically commit the change using the same message than the exported commit appended with the

 (transplanted from REV)

line. If the merge failed, then you have to resolve the conflict and commit as usual. See also the transplant extension doc for more details.

Instead of the transplant command, you might also do:

 $ hg export REV | hg import -

or even:

 $ hg export REV | hg import - -m "backporting REV : short explanation"

Commit review

Unless we do not find the "optimal" solution for commit reviews here is a possibility using bitbucket.org forks.

Assuming you have some changes in your Eigen tip you want somebody to review, you may proceed as follows.

First, you locally commit your changes in your cloned working copy of Eigen

 $ cd eigen # the cloned working copy
 $ hg commit -m "My meaningful commit message."
 $ cd ..

Afterwards, you need to create a fork of Eigen. Once the fork is created you can clone it

 $ hg clone https://login@bitbucket.org/login/eigen2/ eigen2_clone

The next step is to pull the changes from the local clone of your Eigen tip into the forked version.

 $ cd eigen2_clone
 $ hg pull -u -r tip ../eigen

Finally, you can push back the potential commit to the Eigen fork via

 $ hg push

Your commit is now publicly visible and the developers can review it.

Receiving commit notification

The Bitbucket web interface offers RSS and Atom feeds.

If you prefer email notification, subscribe to the eigen-commits mailing list. Instructions are the same as for the eigen mailing list, just replace "eigen" by "eigen-commits" everywhere.

Search and replace on whole directory

If you do a search-and-replace operation on a whole directory, you probably do not want it to affect Mercurial's own internal files! So you must avoid all .hg directories. You can do:

 $ find . -type f | grep -v .hg | xargs sed -i 's/oldstring/newstring/g'

Make a new release

These days it's very easy, as the docs are automatically generated and uploaded by Thomas' machine, and the tarballs are auto-generated by Bitbucket.

So here's what it takes to make a new release (a new minor version):

  • Announce a week in advance on the eigen mailing list the upcoming release (give a precise date) so that people can test and/or speak up if they know of an issue. Of course, in case of an emergency, no need to do that :)
  • Concert with other devs, and/or use CDash, and/or test yourself, to make sure that the test suite passes at the very least for:
    • latest stable GCC
    • some older GCC 4.x (ok, it's hard to check them all)
    • oldest GCC we're supposed to support (currently 3.3)
    • latest MSVC
  • Also do think to check "make install"! Remember how it spoiled the 2.0.7 release!
    • Supreme refinement would be to check building the test suite against the installed Eigen.
  • Update the version number (EIGEN_MINOR_VERSION, etc) in the file Eigen/src/Core/util/Macros.h.
  • In the 2.0 branch, also update the version number in the root CMakeLists.txt. That doesn't apply to the default branch.
  • commit that:
$ hg commit
  • Make a tag:
$ hg tag 4.5.6-beta7
  • recommended: check that everything is in order.
$ hg view
  • push the release:
$ hg push
  • For a minor version: make the full changelog. It's not just a dump of the commit messages: it must be readable and interesting for Eigen users. You can use "hg view" again, try to put the grave bug fixes first, try to group together what's related.
  • Update the Main Page on the wiki, that is the announcement at the top and the Download section. Make sure that the tarball links point to the new release, and that any example of how to get the latest tag also refers to the new tag.
  • Add the release notes into the ChangeLog page.
  • Write a mail to the eigen mailing list, paste the changelog into it.
  • Update the #eigen channel topic on IRC.

Debugging tips

Debugging under Visual Studio

"Examplary view of the enhanced debugger output."

The Visual Studio IDE allows to add specialized debug visualizers for C/C++ code. The way in which specific classes are displayed in the debug output is defined by the file

   %VS90COMNTOOLS%\..\Packages\Debugger\autoexp.dat 

(note that the environment variable may change depending on your IDE version, e.g. VS80COMMONTOOLS).

Adding the data found in this file right after the [Visualizer] section activates new visualization rules for some Eigen types. Currently, all kinds of Matrix types are covered wheras e.g. Eigen::SparseMatrix and Eigen::Transform are still missing. Any new visualizers are welcome and will be added with a note referencing the author to our current file.

Before working on the autoexp.dat file please ensure to create a backup copy!

Studying assembly output

We have a macro EIGEN_ASM_COMMENT (currently defined only on GCC! please port it, it is in Macros.h):

You can use assembly comments to make it much easier to find the asm code corresponding to a particular portion of your code:

 EIGEN_ASM_COMMENT("begin");
 Vector4f u = v + 3*w;
 EIGEN_ASM_COMMENT("end");

Then tell your compiler to output assembly code, for example with GCC it is the -S option, so you'd do:

 g++ myprogram.cpp -O2 -S -o myprogram.s

The resulting asm code is now very easy to find, just search for "begin":

 #APP
 # 5 "x.cpp" 1
 	#begin
 # 0 "" 2
 #NO_APP
 	movss	.LC2(%rip), %xmm0
 	shufps	$0, %xmm0, %xmm0
 	mulps	(%rdx), %xmm0
 	addps	(%rsi), %xmm0
 	movaps	%xmm0, (%rdi)
 #APP
 # 7 "x.cpp" 1
 	#end
 # 0 "" 2
 #NO_APP

Profiling tips

Using timers

In the file bench/BenchTimer.h we have a little timer that can be used for that.

Documentation tips

We use Doxygen, see the commands. See existing source code for example usage. Here are some best practices:

  • Make generous use of "see also" tags: \sa
  • Always use tags for parameters and return values (e.g. \param and \returns), that makes things easier to find in a glance.
  • If possible, add a code snippet. All you have is to add it in doc/snippets (see existing files there). CMake will complete your snippet into working form, compile it, run it, and record its output into a .out file. Then in your doxygen comment, just put:
 \include mysnippet.cpp
 Output: \verbinclude mysnippet.out
  • Alternatively, if you want to add a full self-compilable example (so you dont want CMake to add stuff to make it compilable), put your example file in doc/examples.