Created attachment 736 [details]
Compile this file with Eigen on the include search path. Then run on the command line
When applying an axis-aligned scale (using `DiagonalMatrix`) to a `Transform` object whose `Mode` template argument is set to `Projective`, and whose lowest row elements are non-zero, the result is incorrect. That is, the scaling will not be applied to the elements in the final row. I'm assuming this is just a mis-assumption about the matrix, since this is technically correct for an `Affine` transform type.
Attached is an example demonstrating this. If I apply the `DiagonalMatrix` directly to a particular `Transform`, I get a different result than if I convert the `DiagonalMatrix` to a `Transform`, and then apply it to the same transform as before, I get different results.
I don't think this should be terribly difficult to solve, and if I have the time, I might look into fixing it, I'm just unfamiliar with the internals of Eigen, so I'm sure someone with a little more familiarity could solve it in a few minutes.
This file should compile with the following (with /usr/local/include/eigen3 replaced with whatever your install dir for Eigen is):
g++ -I/usr/local/include/eigen3 eigen_diagonal_affine_scale_problem.cpp -o eigen_diagonal_affine_scale_problem
And run with:
I have only tested this on Linux so far, however I doubt it wouldn't show up on any other platform.
- Stephen Kiazyk
Thank you for catching that one. Fixed:
Summary: Bug 1304: fix Projective * scaling and Projective *= scaling
Backport to 3.2: