1454838 - Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_screenshot.py TestScreenCaptureContent.test_scroll_no_element | application crashed [@ dalvik-jit-code-cache (deleted) + 0x19d44]

Status: RESOLVED DUPLICATE of bug 1454680

(Testing :: Marionette Client and Harness, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ncsoregi [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=174210615&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/AxnJmTZdTEC1tQo74qYCdg/runs/0/artifacts/public/logs/live_backing.log

[task 2018-04-17T23:20:23.011Z] 23:20:23     INFO -  TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_screenshot.py TestScreenCaptureContent.test_scroll_no_element
[task 2018-04-17T23:20:42.513Z] 23:20:42     INFO -  mozcrash Copy/paste: /usr/local/bin/linux64-minidump_stackwalk /tmp/tmpcz00fz/651ead2a-c4ae-ded2-7de1-0929b594b50a.dmp /builds/worker/workspace/build/symbols
[task 2018-04-17T23:20:52.208Z] 23:20:52     INFO -  mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/651ead2a-c4ae-ded2-7de1-0929b594b50a.dmp
[task 2018-04-17T23:20:52.209Z] 23:20:52     INFO -  mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/651ead2a-c4ae-ded2-7de1-0929b594b50a.extra
[task 2018-04-17T23:20:52.212Z] 23:20:52  WARNING -  PROCESS-CRASH | testing/marionette/harness/marionette_harness/tests/unit/test_screenshot.py TestScreenCaptureContent.test_scroll_no_element | application crashed [@ dalvik-jit-code-cache (deleted) + 0x19d44]
[task 2018-04-17T23:20:52.213Z] 23:20:52     INFO -  Crash dump filename: /tmp/tmpcz00fz/651ead2a-c4ae-ded2-7de1-0929b594b50a.dmp
[task 2018-04-17T23:20:52.213Z] 23:20:52     INFO -  Operating system: Android
[task 2018-04-17T23:20:52.213Z] 23:20:52     INFO -                    0.0.0 Linux 2.6.29-gea477bb #1 Wed Sep 26 11:04:45 PDT 2012 armv7l
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -  CPU: arm
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -       ARMv7 ARM Cortex-A8 features: swp,half,thumb,fastmult,vfpv2,edsp,neon,vfpv3
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -       1 CPU
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -  GPU: UNKNOWN
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -  Crash reason:  SIGSEGV
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -  Crash address: 0x4d502d8a
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -  Process uptime: not available
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -  Thread 0 (crashed)
[task 2018-04-17T23:20:52.219Z] 23:20:52     INFO -   0  dalvik-jit-code-cache (deleted) + 0x19d44
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -       r0 = 0x4c527a60    r1 = 0x4c466d50    r2 = 0x00000004    r3 = 0x4d7cd000
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -       r4 = 0x4d502d86    r5 = 0x4c466d50    r6 = 0x2a00d090    r7 = 0x00000004
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -       r8 = 0x00000000    r9 = 0x41525560   r10 = 0x4c466d2c   r12 = 0x55f19d1d
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -       fp = 0x00000014    sp = 0xbee2c8b0    lr = 0x55f19d1b    pc = 0x55f19d44
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -      Found by: given as instruction pointer in context
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -   1  libdvm.so + 0x2de9a
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -       sp = 0xbee2c8bc    pc = 0x4086be9c
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -      Found by: stack scanning
[task 2018-04-17T23:20:52.220Z] 23:20:52     INFO -   2  dalvik-LinearAlloc (deleted) + 0x69ce
[task 2018-04-17T23:20:52.221Z] 23:20:52     INFO -       sp = 0xbee2c8d0    pc = 0x4c46d9d0
[task 2018-04-17T23:20:52.221Z] 23:20:52     INFO -      Found by: stack scanning

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

From the logcat:

04-17 16:20:29.802   736   736 D GeckoBrowserApp: BrowserApp.onTabChanged: 0: STOP
04-17 16:20:30.272   736   764 I Gecko   : 1524007230278	Marionette	TRACE	91 <- [1,11,null,{}]
04-17 16:20:31.112   736   764 I Gecko   : 1524007231118	Marionette	TRACE	91 -> [0,12,"WebDriver:ExecuteScript",{"scriptTimeout":null,"newSandbox":true,"args":[],"filename":"tests/testing/marionette/harness/marionette_harness/tests/unit/test_screenshot.py","script":"return window.pageYOffset","sandbox":"default","line":63}]
04-17 16:20:31.412   736   764 I Gecko   : 1524007231418	Marionette	TRACE	91 <- [1,12,null,{"value":0}]
04-17 16:20:31.452   736   764 I Gecko   : 1524007231456	Marionette	TRACE	91 -> [0,13,"WebDriver:TakeScreenshot",{"highlights":null,"full":true,"hash":true,"id":null,"scroll":true}]
04-17 16:20:36.822   736   764 I Gecko   : [GFX1-]: Failed to create a SkiaGL DrawTarget, falling back to software
04-17 16:20:36.822   736   764 I Gecko   : 
04-17 16:20:39.122   736   736 E MessageQueue-JNI: Exception in MessageQueue callback: dispatchVsync
04-17 16:20:39.151   736   736 E MessageQueue-JNI: android.util.AndroidRuntimeException: { what=0 when=-17ms } This message is already in use.
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.os.MessageQueue.enqueueMessage(MessageQueue.java:300)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.os.Handler.enqueueMessage(Handler.java:623)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.os.Handler.sendMessageAtTime(Handler.java:592)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.view.Choreographer$FrameDisplayEventReceiver.onVsync(Choreographer.java:729)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.view.DisplayEventReceiver.dispatchVsync(DisplayEventReceiver.java:139)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.os.MessageQueue.nativePollOnce(Native Method)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.os.MessageQueue.next(MessageQueue.java:132)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.os.Looper.loop(Looper.java:124)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at android.app.ActivityThread.main(ActivityThread.java:5103)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at java.lang.reflect.Method.invokeNative(Native Method)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at java.lang.reflect.Method.invoke(Method.java:525)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
04-17 16:20:39.151   736   736 E MessageQueue-JNI: 	at dalvik.system.NativeStart.main(Native Method)
04-17 16:20:39.151   736   736 D AndroidRuntime: Shutting down VM
04-17 16:20:39.161   736   736 W dalvikvm: threadid=1: thread exiting with uncaught exception (group=0x414c8700)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: >>> REPORTING UNCAUGHT EXCEPTION FROM THREAD 1 ("main")
04-17 16:20:39.191   736   736 E GeckoCrashHandler: android.util.AndroidRuntimeException: { what=0 when=-17ms } This message is already in use.
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.os.MessageQueue.enqueueMessage(MessageQueue.java:300)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.os.Handler.enqueueMessage(Handler.java:623)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.os.Handler.sendMessageAtTime(Handler.java:592)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.view.Choreographer$FrameDisplayEventReceiver.onVsync(Choreographer.java:729)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.view.DisplayEventReceiver.dispatchVsync(DisplayEventReceiver.java:139)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.os.MessageQueue.nativePollOnce(Native Method)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.os.MessageQueue.next(MessageQueue.java:132)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.os.Looper.loop(Looper.java:124)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at android.app.ActivityThread.main(ActivityThread.java:5103)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at java.lang.reflect.Method.invokeNative(Native Method)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at java.lang.reflect.Method.invoke(Method.java:525)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:737)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
04-17 16:20:39.191   736   736 E GeckoCrashHandler: 	at dalvik.system.NativeStart.main(Native Method)
04-17 16:20:39.262   736   736 W google-breakpad: ExceptionHandler::GenerateDump cloned child

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Snorp, could this be graphics related? 

04-17 16:20:36.822   736   764 I Gecko   : [GFX1-]: Failed to create a SkiaGL DrawTarget, falling back to software
04-17 16:20:36.822   736   764 I Gecko   : 
04-17 16:20:39.122   736   736 E MessageQueue-JNI: Exception in MessageQueue callback: dispatchVsync

Comment 3

[James Willcox (:snorp) ([e-mail]) (he/him)]

I think that message is pretty common during reftests because there is a cap on the number of SkiaGL canvases we can create. Marionette probably has the same problem if it's using the canvas to create snapshots. At one point we were just going to disable skiagl for those kinds of tests, but I can't remember what happened there.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+2]]

That sounds plausible. Maybe we could try to do the same for the Marionette tests. Do you know how I can disable skiagl?

Comment 5

[James Willcox (:snorp) ([e-mail]) (he/him)]

(In reply to Henrik Skupin (:whimboo) from comment #4)
> That sounds plausible. Maybe we could try to do the same for the Marionette
> tests. Do you know how I can disable skiagl?

gfx.canvas.azure.accelerated = false

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I pushed a try build which globally disables it for Fennec in Marionette tests. Lets see:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b32063aafefb67f2e635bfa9f336661ac51ee7fc

Note that I cannot find any other harness in the tree which disables this flag.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

For reftests we disabled skigl for a while but that got re-enabled via bug 1298023. Not sure what might be different for Marionette, but I would wait with disabling it and observe test failures for a while. Lets see how often it fails.

Comment 8

[Intermittent Failures Robot]

20 failures in 167 pushes (0.12 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* try: 19
* mozilla-beta: 1

Platform breakdown:
* android-4-3-armv7-api16: 20

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1454838&startday=2018-04-19&endday=2018-04-19&tree=trunk

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

To note the above 19 failures on try were all from the beta simulation as run yesterday. I will wait for today's results given that have one other bug fix, and more in depth Marionette logs available on Android.

Comment 10

[Intermittent Failures Robot]

36 failures in 782 pushes (0.046 failures/push) were associated with this bug in the last 7 days. 

This is the #49 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* try: 34
* mozilla-inbound: 1
* mozilla-beta: 1

Platform breakdown:
* android-4-3-armv7-api16: 36

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1454838&startday=2018-04-16&endday=2018-04-22&tree=trunk

Comment 11

[Intermittent Failures Robot]

2 failures in 780 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 1
* mozilla-inbound: 1

Platform breakdown:
* android-4-3-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1454838&startday=2018-04-23&endday=2018-04-29&tree=trunk

Comment 12

[Intermittent Failures Robot]

1 failures in 551 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 1

Platform breakdown:
* android-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1454838&startday=2018-04-30&endday=2018-05-06&tree=trunk

Comment 13

[Intermittent Failures Robot]

4 failures in 660 pushes (0.006 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 3
* mozilla-beta: 1

Platform breakdown:
* android-4-3-armv7-api16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1454838&startday=2018-05-07&endday=2018-05-13&tree=trunk

Comment 14

[Intermittent Failures Robot]

4 failures in 639 pushes (0.006 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 2
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* android-4-3-armv7-api16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1454838&startday=2018-05-14&endday=2018-05-20&tree=trunk

Comment 15

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Maybe we should just dupe it against bug 1454680.

Comment 16

[Intermittent Failures Robot]

3 failures in 632 pushes (0.005 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 1
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* android-4-3-armv7-api16: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1454838&startday=2018-05-21&endday=2018-05-27&tree=trunk

Comment 17

[BMO Automation]

Moving bug to Testing::Marionette Client and Harness component per bug 1815831.