1363092 - Intermittent test_discardAnimatedImage.html | application crashed [@ mozilla::image::RasterImage::Decode]

Status: RESOLVED FIXED

(Core :: Graphics: ImageLib, defect)

Description

[Treeherder Bug Filer]

Filed by: wkocher [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=97245210&repo=autoland

https://queue.taskcluster.net/v1/task/QdwJlPsRQxi6jhZR778oaw/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Andrew Osmond [:aosmond] (he/him)]

Stack trace from the log:

[task 2017-05-08T07:44:52.834701Z] 07:44:52  WARNING -  PROCESS-CRASH | image/test/mochitest/test_discardAnimatedImage.html | application crashed [@ mozilla::image::RasterImage::Decode]
[task 2017-05-08T07:44:52.835244Z] 07:44:52     INFO -  Crash dump filename: /tmp/tmpRJw3nu/7f0052f7-bca2-9429-0be0-5a47cb198212.dmp
[task 2017-05-08T07:44:52.835419Z] 07:44:52     INFO -  Operating system: Android
[task 2017-05-08T07:44:52.835753Z] 07:44:52     INFO -                    0.0.0 Linux 2.6.29-gea477bb #1 Wed Sep 26 11:04:45 PDT 2012 armv7l
[task 2017-05-08T07:44:52.836200Z] 07:44:52     INFO -  CPU: arm
[task 2017-05-08T07:44:52.842580Z] 07:44:52     INFO -       ARMv7 ARM Cortex-A8 features: swp,half,thumb,fastmult,vfpv2,edsp,neon,vfpv3
[task 2017-05-08T07:44:52.842632Z] 07:44:52     INFO -       1 CPU
[task 2017-05-08T07:44:52.842677Z] 07:44:52     INFO -  GPU: UNKNOWN
[task 2017-05-08T07:44:52.842719Z] 07:44:52     INFO -  Crash reason:  SIGSEGV
[task 2017-05-08T07:44:52.842755Z] 07:44:52     INFO -  Crash address: 0x0
[task 2017-05-08T07:44:52.842793Z] 07:44:52     INFO -  Process uptime: not available
[task 2017-05-08T07:44:52.842828Z] 07:44:52     INFO -  Thread 12 (crashed)
[task 2017-05-08T07:44:52.842886Z] 07:44:52     INFO -   0  libxul.so!mozilla::image::RasterImage::Decode [RasterImage.cpp:676720d93349 : 1278 + 0x0]
[task 2017-05-08T07:44:52.842943Z] 07:44:52     INFO -       r0 = 0x0000006f    r1 = 0xa6b9c6ea    r2 = 0xa6b9c6ea    r3 = 0xa6b9c6ea
[task 2017-05-08T07:44:52.842997Z] 07:44:52     INFO -       r4 = 0x55dfd160    r5 = 0x00000000    r6 = 0x00000000    r7 = 0x00000025
[task 2017-05-08T07:44:52.843062Z] 07:44:52     INFO -       r8 = 0x00000000    r9 = 0x00000001   r10 = 0x529aaa20   r12 = 0x00000003
[task 2017-05-08T07:44:52.843119Z] 07:44:52     INFO -       fp = 0x55dfd19c    sp = 0x529aaa08    lr = 0x5ac511bb    pc = 0x5ac63d96
[task 2017-05-08T07:44:52.843170Z] 07:44:52     INFO -      Found by: given as instruction pointer in context
[task 2017-05-08T07:44:52.843231Z] 07:44:52     INFO -   1  libxul.so!mozilla::image::RasterImage::LookupFrame [RasterImage.cpp:676720d93349 : 354 + 0xb]
[task 2017-05-08T07:44:52.843296Z] 07:44:52     INFO -       r4 = 0x529aaad0    r5 = 0x55dfd160    r6 = 0x00000025    r7 = 0x529aaa68
[task 2017-05-08T07:44:52.843359Z] 07:44:52     INFO -       r8 = 0x529aaa60    r9 = 0x00000001   r10 = 0x00000001    fp = 0x529aae48
[task 2017-05-08T07:44:52.843411Z] 07:44:52     INFO -       sp = 0x529aaa58    pc = 0x5ac64a3d
[task 2017-05-08T07:44:52.843458Z] 07:44:52     INFO -      Found by: call frame info
[task 2017-05-08T07:44:52.843549Z] 07:44:52     INFO -   2  libxul.so!mozilla::image::RasterImage::Draw [RasterImage.cpp:676720d93349 : 1475 + 0x11]
[task 2017-05-08T07:44:52.843607Z] 07:44:52     INFO -       r4 = 0x00000025    r5 = 0x55dfd160    r6 = 0x529aaad0    r7 = 0x00000001
[task 2017-05-08T07:44:52.843662Z] 07:44:52     INFO -       r8 = 0x529aae40    r9 = 0x55d54040   r10 = 0x00000000    fp = 0x529aae48
[task 2017-05-08T07:44:52.843706Z] 07:44:52     INFO -       sp = 0x529aaab0    pc = 0x5ac6654b
[task 2017-05-08T07:44:52.843753Z] 07:44:52     INFO -      Found by: call frame info

which points to:

https://hg.mozilla.org/integration/autoland/file/676720d93349c3505498f0ad06478c041d04fd2a/image/RasterImage.cpp#l1278

which doesn't seem promising given rect is an object (not a pointer) on the stack, we check mAnimationState in the if statement just before, mSourceBuffer is created at RasterImage construction time and task is also checked further below.

Comment 2

[Andrew Osmond [:aosmond] (he/him)]

(In reply to Andrew Osmond [:aosmond] from comment #1)
> Stack trace from the log:
> 
> [task 2017-05-08T07:44:52.834701Z] 07:44:52  WARNING -  PROCESS-CRASH |
> image/test/mochitest/test_discardAnimatedImage.html | application crashed [@
> mozilla::image::RasterImage::Decode]
> [task 2017-05-08T07:44:52.835244Z] 07:44:52     INFO -  Crash dump filename:
> /tmp/tmpRJw3nu/7f0052f7-bca2-9429-0be0-5a47cb198212.dmp
> [task 2017-05-08T07:44:52.835419Z] 07:44:52     INFO -  Operating system:
> Android
> [task 2017-05-08T07:44:52.835753Z] 07:44:52     INFO -                   
> 0.0.0 Linux 2.6.29-gea477bb #1 Wed Sep 26 11:04:45 PDT 2012 armv7l
> [task 2017-05-08T07:44:52.836200Z] 07:44:52     INFO -  CPU: arm
> [task 2017-05-08T07:44:52.842580Z] 07:44:52     INFO -       ARMv7 ARM
> Cortex-A8 features: swp,half,thumb,fastmult,vfpv2,edsp,neon,vfpv3
> [task 2017-05-08T07:44:52.842632Z] 07:44:52     INFO -       1 CPU
> [task 2017-05-08T07:44:52.842677Z] 07:44:52     INFO -  GPU: UNKNOWN
> [task 2017-05-08T07:44:52.842719Z] 07:44:52     INFO -  Crash reason: 
> SIGSEGV
> [task 2017-05-08T07:44:52.842755Z] 07:44:52     INFO -  Crash address: 0x0
> [task 2017-05-08T07:44:52.842793Z] 07:44:52     INFO -  Process uptime: not
> available
> [task 2017-05-08T07:44:52.842828Z] 07:44:52     INFO -  Thread 12 (crashed)
> [task 2017-05-08T07:44:52.842886Z] 07:44:52     INFO -   0 
> libxul.so!mozilla::image::RasterImage::Decode [RasterImage.cpp:676720d93349
> : 1278 + 0x0]
> [task 2017-05-08T07:44:52.842943Z] 07:44:52     INFO -       r0 = 0x0000006f
> r1 = 0xa6b9c6ea    r2 = 0xa6b9c6ea    r3 = 0xa6b9c6ea
> [task 2017-05-08T07:44:52.842997Z] 07:44:52     INFO -       r4 = 0x55dfd160
> r5 = 0x00000000    r6 = 0x00000000    r7 = 0x00000025
> [task 2017-05-08T07:44:52.843062Z] 07:44:52     INFO -       r8 = 0x00000000
> r9 = 0x00000001   r10 = 0x529aaa20   r12 = 0x00000003
> [task 2017-05-08T07:44:52.843119Z] 07:44:52     INFO -       fp = 0x55dfd19c
> sp = 0x529aaa08    lr = 0x5ac511bb    pc = 0x5ac63d96
> [task 2017-05-08T07:44:52.843170Z] 07:44:52     INFO -      Found by: given
> as instruction pointer in context
> [task 2017-05-08T07:44:52.843231Z] 07:44:52     INFO -   1 
> libxul.so!mozilla::image::RasterImage::LookupFrame
> [RasterImage.cpp:676720d93349 : 354 + 0xb]
> [task 2017-05-08T07:44:52.843296Z] 07:44:52     INFO -       r4 = 0x529aaad0
> r5 = 0x55dfd160    r6 = 0x00000025    r7 = 0x529aaa68
> [task 2017-05-08T07:44:52.843359Z] 07:44:52     INFO -       r8 = 0x529aaa60
> r9 = 0x00000001   r10 = 0x00000001    fp = 0x529aae48
> [task 2017-05-08T07:44:52.843411Z] 07:44:52     INFO -       sp = 0x529aaa58
> pc = 0x5ac64a3d
> [task 2017-05-08T07:44:52.843458Z] 07:44:52     INFO -      Found by: call
> frame info
> [task 2017-05-08T07:44:52.843549Z] 07:44:52     INFO -   2 
> libxul.so!mozilla::image::RasterImage::Draw [RasterImage.cpp:676720d93349 :
> 1475 + 0x11]
> [task 2017-05-08T07:44:52.843607Z] 07:44:52     INFO -       r4 = 0x00000025
> r5 = 0x55dfd160    r6 = 0x529aaad0    r7 = 0x00000001
> [task 2017-05-08T07:44:52.843662Z] 07:44:52     INFO -       r8 = 0x529aae40
> r9 = 0x55d54040   r10 = 0x00000000    fp = 0x529aae48
> [task 2017-05-08T07:44:52.843706Z] 07:44:52     INFO -       sp = 0x529aaab0
> pc = 0x5ac6654b
> [task 2017-05-08T07:44:52.843753Z] 07:44:52     INFO -      Found by: call
> frame info
> 
> which points to:
> 
> https://hg.mozilla.org/integration/autoland/file/
> 676720d93349c3505498f0ad06478c041d04fd2a/image/RasterImage.cpp#l1278
> 
> which doesn't seem promising given rect is an object (not a pointer) on the
> stack, we check mAnimationState in the if statement just before,
> mSourceBuffer is created at RasterImage construction time and task is also
> checked further below.

My bad. I thought the assert log was missing so I was searching for a null deref :). As it turns out, the assertion prints 3 seconds *after* the process crash rather than being there *before* -- my first time seeing this.

[task 2017-05-08T07:44:55.242388Z] 07:44:55     INFO -  05-08 00:44:17.501 F/MOZ_Assert(  785): Assertion failure: rect.IsEmpty(), at /home/worker/workspace/build/src/image/RasterImage.cpp:1278

Comment 3

[Andrew Osmond [:aosmond] (he/him)]

Assert was added in bug 1360572.

Comment 4

[Timothy Nikkel (:tnikkel)]

The problem is that we are doing the drawWindow call from the test, which asks for sync decode of images. We already have a decode started at this point. So when we lookup the frame and ask the FrameAnimator for the composited frame it returns pending. But because it is sync decode we proceed to call Decode to launch another decode. When the UpdateState call happens it determines that we are fully decoded because it can seek to the last frame. And that's how we get the non-empty rect. Thinking about the best way to fix this.

Comment 5

[Intermittent Failures Robot]

37 failures in 879 pushes (0.042 failures/push) were associated with this bug in the last 7 days. 

This is the #37 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-inbound: 20
* autoland: 15
* try: 1
* mozilla-central: 1

Platform breakdown:
* android-4-3-armv7-api15: 37

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1363092&startday=2017-05-08&endday=2017-05-14&tree=all

Comment 6

[Intermittent Failures Robot]

30 failures in 777 pushes (0.039 failures/push) were associated with this bug in the last 7 days. 

This is the #47 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 20
* mozilla-inbound: 7
* try: 1
* mozilla-central: 1
* graphics: 1

Platform breakdown:
* android-4-3-armv7-api15: 30

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1363092&startday=2017-05-15&endday=2017-05-21&tree=all

Comment 7

[Geoff Brown [:gbrown]]

No failures since May 20. I don't see any changes to the test.

Comment 8

[Timothy Nikkel (:tnikkel)]

Attachment: patch

This is probably not the best way to fix this, but it is a way. The other ways involve making larger changes.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/9d8e1c9e19e0
Don't update the state of an animated image that requires an invalidation when creating a new decoder because we may not be able to send invalidations. r=aosmond

Comment 10

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/9d8e1c9e19e0

Comment 11

[Intermittent Failures Robot]

2 failures in 814 pushes (0.002 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api15: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1363092&startday=2017-06-12&endday=2017-06-18&tree=all

Comment 12

[Timothy Nikkel (:tnikkel)]

(In reply to OrangeFactor Robot from comment #11)
> 2 failures in 814 pushes (0.002 failures/push) were associated with this bug
> in the last 7 days.   
> 
> Repository breakdown:
> * mozilla-inbound: 1
> * autoland: 1
> 
> Platform breakdown:
> * android-4-3-armv7-api15: 2
> 
> For more details, see:
> https://brasstacks.mozilla.com/orangefactor/
> ?display=Bug&bugid=1363092&startday=2017-06-12&endday=2017-06-18&tree=all

Filed bug 1374123 to fix the silly mistake from the patch here.