Created attachment 557 [details]
In the attached test, the case the norms are very large when the the product of the factors is subtracted from the original input matrices. In the hessenbergTriangular() method, the code tests for whether an element of the input matrix is zero. If it is zero, then the Givens rotation is not applied to S or T. However, if m_computeQZ is true, it applies the Givens rotation to Q or Z regardless of whether S or T was rotated. The solution in the patch is to move the rotation for Q or Z inside the if(coeff(i,j) != 0) block. This makes the norms zero to machine precision.
Created attachment 558 [details]
patch to fix bug
Thanks for the report and the patch.
Fixed in devel and 3.2 branch:
FYI: The changelog entry for bug 985 says "fix RealQZ when either matrix had zero rows or columns", which is not the correct description.
Well, that description was indeed very ambiguous. I meant to say "zero rows" as in "at least one zero row", i.e. a row containing only zeros -- having just single zeros was no problem, IIRC.
(This is also just FYI, changing the commit message now likely will mess up several things)