This bugzilla service is closed. All entries have been migrated to https://gitlab.com/libeigen/eigen
Bug 1449 - redux_3 test case generates unnecessary temporary on SLES-12 ppc64le
Summary: redux_3 test case generates unnecessary temporary on SLES-12 ppc64le
Status: RESOLVED FIXED
Alias: None
Product: Eigen
Classification: Unclassified
Component: Core - vectorization (show other bugs)
Version: 3.3 (current stable)
Hardware: PPC - general Linux
: Normal Failed Unit Test
Assignee: Konstantinos Margaritis
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-17 06:08 UTC by Yugandha Deshpande
Modified: 2019-12-04 17:06 UTC (History)
5 users (show)



Attachments
Error log (793 bytes, text/plain)
2017-07-17 09:38 UTC, Yugandha Deshpande
no flags Details

Description Yugandha Deshpande 2017-07-17 06:08:03 UTC
Hi,

I am trying to build and test eigen 3.3.4 on SLES-12 ppc64le, with gcc version 6.3.1. I am getting following test failures:

99% tests passed, 11 tests failed out of 772

Label Time Summary:
Official       = 14060.44 sec (661 tests)
Unsupported    = 3240.67 sec (110 tests)

Total Test time (real) = 17446.42 sec

The following tests FAILED:
         74 - redux_3 (OTHER_FAULT)
        383 - qr_colpivoting_1 (OTHER_FAULT)
        480 - bdcsvd_9 (OTHER_FAULT)
        569 - sparse_basic_2 (Timeout)
        661 - fastmath (OTHER_FAULT)
        685 - matrix_exponential_9 (OTHER_FAULT)
        695 - matrix_power_9 (OTHER_FAULT)
        698 - matrix_power_12 (Timeout)
        714 - mpreal_support (OTHER_FAULT)
        717 - sparse_extra_3 (OTHER_FAULT)
        766 - cxx11_tensor_io (SEGFAULT)
Errors while running CTest
make[3]: *** [CMakeFiles/check.dir/build.make:57: CMakeFiles/check] Error 8
make[2]: *** [CMakeFiles/Makefile2:356: CMakeFiles/check.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:363: CMakeFiles/check.dir/rule] Error 2
make: *** [Makefile:290: check] Error 2

Has anyone seen the similar failures before? If yes any pointers would help to resolve this.
Comment 1 Yugandha Deshpande 2017-07-17 09:38:53 UTC
Created attachment 793 [details]
Error log
Comment 2 Christoph Hertzberg 2017-08-02 11:31:04 UTC
I renamed the bug and changed the component, since this seems to be the only relevant problem here.
Please open new bugs for the other failures, if they don't have one already.
Comment 3 Christoph Hertzberg 2017-08-02 12:53:51 UTC
N.B.: Assuming associative math, the failing expression  `(A * B).sum()`  could actually be evaluated as:
  A.colwise().sum() * B.rowwise().sum();

which should always work without temporaries. (But that is an optimization unrelated to this issue)
Comment 4 Yugandha Deshpande 2017-08-04 13:38:03 UTC
Hi,
I found that this test passes with older branches. So I tried to bisect it with 3.3-alpha1 as the good candidate and I got the following result:

The first bad revision is:
changeset:   10323:372711b73f7b
branch:      3.3
parent:      10321:e97d1b192338
user:        Gael Guennebaud <g.gael@free.fr>
date:        Fri Feb 17 14:10:57 2017 +0100
summary:     Bug 1393: enable Matrix/Array explicit ctor from types with convers      ion operators (was ok with 3.2)

So it seems that commit https://bitbucket.org/eigen/eigen/commits/372711b73f7bf2d3d7a3160b966e0f77ef8f04a2 is the reason of redux_3 failure
Comment 5 Konstantinos Margaritis 2017-08-07 09:51:26 UTC
FWIW, this also occurs in the s390x/Zvector arch:

> ./build/test/redux_3 
Initializing random number generator with seed 1502099352
Repeating each test 10 times
nb_temporaries == 1
Test matrixRedux(Array4d()) failed in /srv/data/eigen.mine/test/redux.cpp (73)
    ("(m1.matrix()*m1.matrix().transpose()).sum()") && nb_temporaries==(MatrixType::IsVectorAtCompileTime && MatrixType::SizeAtCompileTime!=1 ? 0 : 1)
Stack:
  - matrixRedux(Array4d())
  - redux
  - Seed: 1502099352

Ακυρώθηκε
Comment 6 Yugandha Deshpande 2017-08-18 06:56:08 UTC
Are you planning to fix this bad commit then?
Comment 7 Gael Guennebaud 2017-08-22 13:57:23 UTC
The problem is not that commit, actually this commit revealed this issue that was hidden so far. The true issue is that on your platform:

EIGEN_CACHEFRIENDLY_PRODUCT_THRESHOLD==4

whereas it is defined to 8 on most platforms.

Then v * v.transpose() with v a Vector4 follows the "OuterProduct" path (instead of a lazy coefficient-based evaluation), which, for efficiency reasons, explicitly evaluate the outer-product within a temporary. So the fix is to enforce EIGEN_CACHEFRIENDLY_PRODUCT_THRESHOLD==8 in the unit tests making this assumption.
Comment 8 Gael Guennebaud 2017-08-22 14:00:18 UTC
Fixed:
https://bitbucket.org/eigen/eigen/commits/4a0cb9e15fbd/
Comment 9 Yugandha Deshpande 2017-08-23 10:41:51 UTC
Thanks! The redux_3 test is passing for me now.
Thanks all for the help.
Comment 10 Konstantinos Margaritis 2017-08-23 10:48:15 UTC
Gael, could you please remind me what exactly this define stands for? I remember setting it to 4 for 32-bit powerpc (altivec), but maybe it's wrong for the ppc64 (and maybe s390x where it's set to 4 as well).
Comment 11 Gael Guennebaud 2017-08-24 08:31:16 UTC
That's a compile-time threshold to say whether a compile-time size can be considered large enough. It is mainly used to switch between a simple coefficient-wise product implementation and the heavy matrix product code. It can only be determined by benchmarking:

Matrix<float,N,N> A, B, C;
C.noalias() += A*B;

for N=4..12

But, now, things are a bit more complicated because we also have a runtime heuristic to fallback from the heavy code to the coeff-based one, that is:

if((rhs.rows()+dst.rows()+dst.cols())<20) ..

This hardcoded threshold has been estimated on an Intel corei7 Haswell. So this means that setting EIGEN_CACHEFRIENDLY_PRODUCT_THRESHOLD to something lower than 7 is kind of useless now. See bug 404 for an helper program to measure this new threshold.
Comment 12 Gael Guennebaud 2017-08-24 08:44:39 UTC
I added some comment and made it configurable in changeset 2e1e9d536760, so if you want to try the helper program, define EIGEN_GEMM_TO_COEFFBASED_THRESHOLD to 2000 first.
Comment 13 Nobody 2019-12-04 17:06:24 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to gitlab.com's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.com/libeigen/eigen/issues/1449.

Note You need to log in before you can comment on or make changes to this bug.