Created attachment 328 [details]
Test-case to trigger the assertion.
If I use a unsigned index type (#define EIGEN_DEFAULT_DENSE_INDEX_TYPE unsigned), all householder QR preconditioners trigger an assertion. Looking at the source it seems that the SVD code doesn't seem to be written with unsigned indexes in mind (for loops while --i > 0, etc).
Created attachment 329 [details]
This compilation log, produces with Clang -Wall -Wextra, lists some of the code constructs which seem incapable of working with unsigned indices.
Created attachment 330 [details]
Backtrace from the triggered assertion.
Index is generally supposed to be a signed integer type (maybe this should be documented where EIGEN_DEFAULT_DENSE_INDEX_TYPE is described).
There may be many parts which also work with unsigned integers, but you can't rely on this always being the case.
Do you have a use case where you need that extra bit for the index type (and where going to the next bigger int-type is not an option)?
No, I just tried to use it since I prefer unsigned types for values which shouldn't go negative (such as matrix dimensions). Certainly no must-have, although it shouldn't fail as horribly as it did.
I guess changing that design decision is hardly possible now. However, it should be possible to add a (more descriptive) static assertion if the user provided index type is not signed.
Date: 2013-06-10 17:16:16
Summary: Fix bug 583: add compile-time check that DenseIndex is signed