822086 - "ASSERTION: Shouldn't be trying to draw with a singular matrix!"

Status: RESOLVED FIXED

(Core :: Graphics, defect)

Description

[Jesse Ruderman]

Attachment: testcase

###!!! ASSERTION: Shouldn't be trying to draw with a singular matrix!: '!aMatrix.IsSingular()', file Layers.h, line 732

This assertion was added in bug 810302.

Comment 1

[Matt Woodrow (:mattwoodrow)]

Interesting.

We create a matrix for the nsDisplayTransform which isn't singular:

(gdb) p newTransformMatrix
$2 = (const gfx3DMatrix &) @0x1123d7ce0: {
  _11 = 2575, 
  _12 = 278, 
  _13 = 3, 
  _14 = 4, 
  _21 = 5153, 
  _22 = 558, 
  _23 = 7, 
  _24 = 8, 
  _31 = 7731, 
  _32 = 838, 
  _33 = 11, 
  _34 = 12, 
  _41 = -1667315.5, 
  _42 = -180007, 
  _43 = -1943.5, 
  _44 = -2590
}

and then call NudgeToIntegers on it in ChooseScaleAndSetTransform:

(gdb) p transform
$3 = {
  _11 = 2575, 
  _12 = 278, 
  _13 = 3, 
  _14 = 4, 
  _21 = 5153, 
  _22 = 558, 
  _23 = 7, 
  _24 = 8, 
  _31 = 7731, 
  _32 = 838, 
  _33 = 11, 
  _34 = 12, 
  _41 = -1667315, 
  _42 = -180007, 
  _43 = -1943.5, 
  _44 = -2590
}

Apparently rounding the _41 value makes this singular.

I guess we should check for singularity again at some point and bail out.

Comment 2

[Matt Woodrow (:mattwoodrow)]

Attachment: Bail out if we end up with a singular matrix

Comment 3

[Matt Woodrow (:mattwoodrow)]

Attachment: Bail out if we end up with a singular matrix v2

Right patch this time hopefully

Comment 4

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 722524 [details] [diff] [review]
Bail out if we end up with a singular matrix v2

Review of attachment 722524 [details] [diff] [review]:
-----------------------------------------------------------------

Can we avoid adding an extra IsSingular check by removing the existing one?

Comment 5

[Matt Woodrow (:mattwoodrow)]

You mean the one in nsDisplayTransform?

Comment 6

[Matt Woodrow (:mattwoodrow)]

Attachment: Bail out if we end up with a singular matrix v3

Comment 7

[Matt Woodrow (:mattwoodrow)]

http://hg.mozilla.org/integration/mozilla-inbound/rev/42694e9f60a7

Comment 8

[Ed Morley [:emorley]]

https://hg.mozilla.org/mozilla-central/rev/42694e9f60a7
https://hg.mozilla.org/mozilla-central/rev/3d83421e9ca4