This bugzilla service is closed. All entries have been migrated to https://gitlab.com/libeigen/eigen
Bug 539 - kronecker_product fails consistently, geo_hyperplane_1 is flaky
Summary: kronecker_product fails consistently, geo_hyperplane_1 is flaky
Status: RESOLVED FIXED
Alias: None
Product: Eigen
Classification: Unclassified
Component: Tests (show other bugs)
Version: 3.1
Hardware: x86 - 64-bit Linux
: Normal Unknown
Assignee: Nobody
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-12-28 19:44 UTC by Ali
Modified: 2019-12-04 12:00 UTC (History)
5 users (show)



Attachments

Description Ali 2012-12-28 19:44:13 UTC
Linux kubuntu 3.2.0-35-generic #55-Ubuntu SMP Wed Dec 5 17:42:16 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) 

----------------------------------------------------------

402/595 Testing: geo_hyperplane_1
402/595 Test: geo_hyperplane_1
Command: "/home/ali/eigen/test/runtest.sh" "geo_hyperplane_1"
Directory: /home/ali/eigen/build_dir/test
"geo_hyperplane_1" start time: Dec 28 19:21 CET
Output:
----------------------------------------------------------
/home/ali/eigen/test/runtest.sh: line 20:  7151 Aborted                 (core dumped) ./$1 > /dev/null 2> .runtest.log
[31m Test geo_hyperplane_1 failed: [30m
[34m
Test hyperplane(Hyperplane<float,2>()) failed in /home/ali/eigen/test/geo_hyperplane.cpp (46)
    test_isApprox(pl1.signedDistance(p1 + n1 * s0), s0)

[30m
<end of output>
Test time =   0.14 sec
----------------------------------------------------------
Test Failed.
"geo_hyperplane_1" end time: Dec 28 19:21 CET
"geo_hyperplane_1" time elapsed: 00:00:00
----------------------------------------------------------

----------------------------------------------------------

592/595 Testing: kronecker_product
592/595 Test: kronecker_product
Command: "/home/ali/eigen/test/runtest.sh" "kronecker_product"
Directory: /home/ali/eigen/build_dir/unsupported/test
"kronecker_product" start time: Dec 28 19:22 CET
Output:
----------------------------------------------------------
/home/ali/eigen/test/runtest.sh: line 20:  7589 Aborted                 (core dumped) ./$1 > /dev/null 2> .runtest.log
[31m Test kronecker_product failed: [30m
[34m
kronecker_product: /home/ali/eigen/Eigen/src/SparseCore/SparseMatrix.h:908: Eigen::SparseMatrix<_Scalar, _Flags, _Index>::Scalar& Eigen::SparseMatrix<_Scalar, _Flags, _Index>::insertUncompressed(Eigen::SparseMatrix<_Scalar, _Flags, _Index>::Index, Eigen::SparseMatrix<_Scalar, _Flags, _Index>::Index) [with _Scalar = double, int _Options = 0, _Index = int, Eigen::SparseMatrix<_Scalar, _Flags, _Index>::Scalar = double, Eigen::SparseMatrix<_Scalar, _Flags, _Index>::Index = int]: Assertion `(p<=startId || m_data.index(p-1)!=inner) && "you cannot insert an element that already exist, you must call coeffRef to this end"' failed.
[30m
<end of output>
Test time =   0.08 sec
----------------------------------------------------------
Test Failed.
"kronecker_product" end time: Dec 28 19:22 CET
"kronecker_product" time elapsed: 00:00:00
----------------------------------------------------------
Comment 1 Jitse Niesen 2013-01-03 19:03:41 UTC
The kronecker_product failure seems to be triggered by changeset 6947411ac1e0 (add an assertion when inserting an already existing element).

Changeset 8454dcaa2d68 should fix that.

I didn't do anything about the geo_hyperplane failure.
Comment 2 Ali 2013-01-03 20:24:33 UTC
Thanks. Unfortunately there are more flaky tests:

----------------------------------------------------------

529/595 Testing: bicgstab_1
529/595 Test: bicgstab_1
Command: "/home/ali/eigen/test/runtest.sh" "bicgstab_1"
Directory: /home/ali/eigen/build_dir/test
"bicgstab_1" start time: Dec 28 19:59 CET
Output:
----------------------------------------------------------
/home/ali/eigen/test/runtest.sh: line 20: 12706 Aborted                 (core dumped) ./$1 > /dev/null 2> .runtest.log
[31m Test bicgstab_1 failed: [30m
[34m
sparse solver testing: solving failed
Test check_sparse_square_solving(bicgstab_colmajor_ilut) failed in /home/ali/eigen/test/sparse_solver.h (39)
    x.isApprox(refX,test_precision<Scalar>())

[30m
<end of output>
Test time =   1.35 sec
----------------------------------------------------------
Test Failed.
"bicgstab_1" end time: Dec 28 19:59 CET
"bicgstab_1" time elapsed: 00:00:01
----------------------------------------------------------

----------------------------------------------------------

595/595 Testing: gmres_2
595/595 Test: gmres_2
Command: "/home/ali/eigen/test/runtest.sh" "gmres_2"
Directory: /home/ali/eigen/build_dir/unsupported/test
"gmres_2" start time: Dec 28 19:59 CET
Output:
----------------------------------------------------------
/home/ali/eigen/test/runtest.sh: line 20: 12898 Aborted                 (core dumped) ./$1 > /dev/null 2> .runtest.log
[31m Test gmres_2 failed: [30m
[34m
Test check_sparse_square_solving(gmres_colmajor_diag) failed in /home/ali/eigen/unsupported/test/../../test/sparse_solver.h (39)
    x.isApprox(refX,test_precision<Scalar>())

[30m
<end of output>
Test time =   1.39 sec
----------------------------------------------------------
Test Failed.
"gmres_2" end time: Dec 28 19:59 CET
"gmres_2" time elapsed: 00:00:01
----------------------------------------------------------
Comment 3 Jitse Niesen 2013-01-04 17:19:27 UTC
The tests are with random matrices, so we expect some tests to fail occasionally. On my computer, both bicgstab_1 and gmres_2 succeed (I tried both tests more than 10 times). 

I have three tests that fail consistently: levenberg_marquardt, minres_1, NonLinearOptimization. The last test has been failing for a very long time.
Comment 4 Jitse Niesen 2013-01-04 17:52:26 UTC
To clarify, if a test fails on your system consistently or frequently, then that is a bug, but I cannot reproduce it at the moment. If a test fails only occasionally, then that is not a priority (though I would prefer tests to fail only very rarely).

The levenberg_marquardt and minres_1 tests have failed on my computer from the moment they were introduced. As they are in unsupported/ I am not bothered at the moment.
Comment 5 Ali 2013-01-04 19:29:24 UTC
Running tests with random matrices has the obvious advantage that you are most likely test a brand new case every time you run a test. This is good.

If you carefully log the seed and use a random number generator that produces the same sequence on all platforms then you can reproduce the test if it fails.

See for example 
  http://stackoverflow.com/q/922358/341970
  http://stackoverflow.com/q/13589215/341970

Should I submit a bug report ask for (a) logging the seed and (b) using consistent, cross-platform random number generators?
Comment 6 Benoit Steiner 2014-01-07 23:08:33 UTC
The geo_hyperplane_1 test fails consistently when run with seed s1389124066:

Eigen3/reference_build$ test/geo_hyperplane_1 s1389124066
Initializing random number generator with seed 1389124066
Repeating each test 10 times
Test lines<float>() failed in /usr/local/google/home/bsteiner/Eigen3/eigen/test/geo_hyperplane.cpp (107)
    test_isApprox(result, center)
Stack:
  - lines<float>()
  - geo_hyperplane
  - Seed: 1389124066

Aborted (core dumped)
Comment 7 Gael Guennebaud 2014-01-11 20:06:14 UTC
regarding geo_hyperplane:

https://bitbucket.org/eigen/eigen/commits/9975efe40387/
Changeset:   9975efe40387
User:        ggael
Date:        2014-01-11 20:04:36
Summary:     Make geo_hyperplane unit test more stable (bug 539)
Comment 8 Christoph Hertzberg 2014-03-17 12:49:08 UTC
Except for gmres_2, which is also covered by bug 715, this seems to be fixed by now, so I'm closing this.

(In reply to comment #5)
> Should I submit a bug report ask for (a) logging the seed and (b) using
> consistent, cross-platform random number generators?

ad a): The seeds is logged in ${build-dir}/Testing/Temporary/LastTest.log
ad b): Gael created a new entry for this: Bug 746
Comment 9 Nobody 2019-12-04 12:00:21 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to gitlab.com's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.com/libeigen/eigen/issues/539.

Note You need to log in before you can comment on or make changes to this bug.