1500977 - Intermittent TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output

Status: RESOLVED INCOMPLETE

(Testing :: General, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: rmaries [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=206986237&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/IGW9E9mJRT620ijF_PmAaw/runs/0/artifacts/public/logs/live_backing.log

[task 2018-10-22T14:41:15.572Z] 14:41:15     INFO -  TEST-START | dom/browser-element/mochitest/test_browserElement_inproc_ReloadPostRequest.html
[task 2018-10-22T14:41:15.573Z] 14:41:15     INFO -  TEST-SKIP | dom/browser-element/mochitest/test_browserElement_inproc_ReloadPostRequest.html | took 0ms
[task 2018-10-22T14:41:15.573Z] 14:41:15     INFO -  Running manifest: devtools/server/socket/tests/chrome.ini
[task 2018-10-22T14:41:16.793Z] 14:41:16     INFO -  adb Ignoring attempt to chmod external storage
[task 2018-10-22T14:41:16.886Z] 14:41:16     INFO -  pushing /builds/worker/workspace/build/tests/mochitest/ to /sdcard/tests/chrome on device...
[task 2018-10-22T14:41:31.378Z] 14:41:31     INFO -  runtests.py | Failed to copy /builds/worker/workspace/build/tests/mochitest/hyphenation to profile
[task 2018-10-22T14:41:31.734Z] 14:41:31     INFO -  pk12util: PKCS12 IMPORT SUCCESSFUL
[task 2018-10-22T14:41:31.886Z] 14:41:31     INFO -  MochitestServer : launching [u'/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/xpcshell', '-g', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64', '-f', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmptWTTDD.mozrunner'; const _SERVER_PORT = '8854'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/builds/worker/workspace/build/tests/mochitest/server.js']
[task 2018-10-22T14:41:31.886Z] 14:41:31     INFO -  runtests.py | Server pid: 1046
[task 2018-10-22T14:41:31.890Z] 14:41:31     INFO -  runtests.py | Websocket server pid: 1049
[task 2018-10-22T14:41:31.902Z] 14:41:31     INFO -  runtests.py | SSL tunnel pid: 1052
[task 2018-10-22T14:41:35.445Z] 14:41:35     INFO -  adb Ignoring attempt to chmod external storage
[task 2018-10-22T14:41:35.445Z] 14:41:35     INFO -  runtests.py | Running with e10s: False
[task 2018-10-22T14:41:35.445Z] 14:41:35     INFO -  runtests.py | Running with serviceworker_e10s: False
[task 2018-10-22T14:41:35.446Z] 14:41:35     INFO -  runtests.py | Running tests: start.
[task 2018-10-22T14:41:35.655Z] 14:41:35     INFO -  adb launch_application: am start -W -n org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp -a android.intent.action.VIEW --es env9 MOZ_CRASHREPORTER_NO_REPORT=1 --es env8 R_LOG_DESTINATION=stderr --es args "-no-remote -profile /sdcard/tests/profile//" --es env3 DISABLE_UNSAFE_CPOW_WARNINGS=1 --es env2 R_LOG_VERBOSE=1 --es env1 XPCOM_DEBUG_BREAK=stack --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_LOG_FILE=/sdcard/tests/mozlog/moz.log --es env6 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env5 MOZ_IN_AUTOMATION=1 --es env4 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env12 MOZ_HIDE_RESULTS_TABLE=1 --es env11 R_LOG_LEVEL=6 --es env10 MOZ_PROCESS_LOG=/tmp/tmpGG4AaRpidlog -d http://mochi.test:8888/redirect.html
[task 2018-10-22T14:41:45.485Z] 14:41:45     INFO -  INFO | automation.py | Application pid: 837
[task 2018-10-22T14:43:14.228Z] 14:43:14     INFO -  0 INFO SimpleTest START
[task 2018-10-22T14:43:14.229Z] 14:43:14     INFO -  1 INFO TEST-START | devtools/server/socket/tests/test_websocket-server.html
[task 2018-10-22T14:43:14.231Z] 14:43:14     INFO -  2 INFO TEST-OK | devtools/server/socket/tests/test_websocket-server.html | took 6053ms
[task 2018-10-22T14:43:14.231Z] 14:43:14     INFO -  3 INFO TEST-START | Shutdown
[task 2018-10-22T14:43:14.232Z] 14:43:14     INFO -  4 INFO Passed:  3
[task 2018-10-22T14:43:14.232Z] 14:43:14     INFO -  5 INFO Failed:  0
[task 2018-10-22T14:43:14.233Z] 14:43:14     INFO -  6 INFO Todo:    0
[task 2018-10-22T14:43:14.234Z] 14:43:14     INFO -  7 INFO Mode:    non-e10s
[task 2018-10-22T14:43:14.234Z] 14:43:14     INFO -  8 INFO Slowest: 6052ms - chrome://mochitests/content/chrome/devtools/server/socket/tests/test_websocket-server.html
[task 2018-10-22T14:43:14.234Z] 14:43:14     INFO -  9 INFO SimpleTest FINISHED
[task 2018-10-22T14:43:18.976Z] 14:43:18     INFO -  wait for org.mozilla.fennec_aurora complete; top activity=com.android.launcher
[task 2018-10-22T14:43:19.282Z] 14:43:19     INFO -  INFO | automation.py | Application ran for: 0:01:43.836290
[task 2018-10-22T14:43:19.282Z] 14:43:19     INFO -  INFO | zombiecheck | Reading PID log: /tmp/tmpGG4AaRpidlog
[task 2018-10-22T14:43:19.799Z] 14:43:19     INFO -  /data/tombstones does not exist; tombstone check skipped
[task 2018-10-22T14:43:20.518Z] 14:43:20     INFO -  Stopping web server
[task 2018-10-22T14:43:20.525Z] 14:43:20     INFO -  Stopping web socket server
[task 2018-10-22T14:43:20.545Z] 14:43:20     INFO -  Stopping ssltunnel
[task 2018-10-22T14:43:20.566Z] 14:43:20     INFO -  leakcheck | refcount logging is off, so leaks can't be detected!
[task 2018-10-22T14:43:20.566Z] 14:43:20     INFO -  runtests.py | Running tests: end.
[task 2018-10-22T14:43:21.425Z] 14:43:21     INFO -  Buffered messages finished
[task 2018-10-22T14:43:21.426Z] 14:43:21     INFO -  Running manifest: devtools/shared/security/tests/chrome/chrome.ini
[task 2018-10-22T14:43:22.748Z] 14:43:22     INFO -  adb Ignoring attempt to chmod external storage
[task 2018-10-22T14:43:22.841Z] 14:43:22     INFO -  runtests.py | Failed to copy /builds/worker/workspace/build/tests/mochitest/hyphenation to profile
[task 2018-10-22T14:43:23.202Z] 14:43:23     INFO -  pk12util: PKCS12 IMPORT SUCCESSFUL
[task 2018-10-22T14:43:23.350Z] 14:43:23     INFO -  MochitestServer : launching [u'/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/xpcshell', '-g', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64', '-f', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpC3o3K0.mozrunner'; const _SERVER_PORT = '8854'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/builds/worker/workspace/build/tests/mochitest/server.js']
[task 2018-10-22T14:43:23.351Z] 14:43:23     INFO -  runtests.py | Server pid: 1244
[task 2018-10-22T14:43:23.358Z] 14:43:23     INFO -  runtests.py | Websocket server pid: 1247
[task 2018-10-22T14:43:23.362Z] 14:43:23     INFO -  runtests.py | SSL tunnel pid: 1250
[task 2018-10-22T14:43:26.915Z] 14:43:26     INFO -  adb Ignoring attempt to chmod external storage
[task 2018-10-22T14:43:26.915Z] 14:43:26     INFO -  runtests.py | Running with e10s: False
[task 2018-10-22T14:43:26.915Z] 14:43:26     INFO -  runtests.py | Running with serviceworker_e10s: False
[task 2018-10-22T14:43:26.916Z] 14:43:26     INFO -  runtests.py | Running tests: start.
[task 2018-10-22T14:43:27.224Z] 14:43:27     INFO -  adb launch_application: am start -W -n org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp -a android.intent.action.VIEW --es env9 MOZ_CRASHREPORTER_NO_REPORT=1 --es env8 R_LOG_DESTINATION=stderr --es args "-no-remote -profile /sdcard/tests/profile//" --es env3 DISABLE_UNSAFE_CPOW_WARNINGS=1 --es env2 R_LOG_VERBOSE=1 --es env1 XPCOM_DEBUG_BREAK=stack --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_LOG_FILE=/sdcard/tests/mozlog/moz.log --es env6 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env5 MOZ_IN_AUTOMATION=1 --es env4 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env12 MOZ_HIDE_RESULTS_TABLE=1 --es env11 R_LOG_LEVEL=6 --es env10 MOZ_PROCESS_LOG=/tmp/tmp1piSvcpidlog -d http://mochi.test:8888/redirect.html
[task 2018-10-22T14:43:35.648Z] 14:43:35     INFO -  INFO | automation.py | Application pid: 1038
[task 2018-10-22T14:50:14.683Z] 14:50:14     INFO -  wait for org.mozilla.fennec_aurora complete; top activity=org.mozilla.fennec_aurora
[task 2018-10-22T14:50:14.988Z] 14:50:14     INFO -  Browser unexpectedly found running. Killing...
[task 2018-10-22T14:50:14.989Z] 14:50:14     INFO -  TEST-INFO | started process screentopng
[task 2018-10-22T14:50:15.341Z] 14:50:15     INFO -  TEST-INFO | screentopng: exit 0
[task 2018-10-22T14:50:29.884Z] 14:50:29  WARNING -  TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output
[task 2018-10-22T14:50:29.884Z] 14:50:29     INFO -  INFO | automation.py | Application ran for: 0:07:02.968389
[task 2018-10-22T14:50:29.884Z] 14:50:29     INFO -  INFO | zombiecheck | Reading PID log: /tmp/tmp1piSvcpidlog

Comment 1

[Intermittent Failures Robot]

7 failures in 2584 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 2
* try: 1
* mozilla-inbound: 4

Platform breakdown:
* android-em-4-3-armv7-api16: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-10-22&endday=2018-10-28&tree=all

Comment 2

[Intermittent Failures Robot]

4 failures in 2978 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* autoland: 1
* mozilla-beta: 1
* mozilla-inbound: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-10-29&endday=2018-11-04&tree=all

Comment 3

[Intermittent Failures Robot]

2 failures in 3007 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 1
* autoland: 1

Platform breakdown:
* windows10-64: 1
* android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-11-05&endday=2018-11-11&tree=all

Comment 4

[Intermittent Failures Robot]

4 failures in 2917 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 2
* mozilla-release: 1
* try: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-11-12&endday=2018-11-18&tree=all

Comment 5

[Intermittent Failures Robot]

1 failures in 2387 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-11-19&endday=2018-11-25&tree=all

Comment 6

[Geoff Brown [:gbrown]]

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=214411958&repo=mozilla-inbound&lineNumber=1577

11-28 17:03:48.950  3340  3357 D GeckoViewNavigation[C]: loadURI: uri=http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests where=1 flags=0 tp=moz-nullprincipal:{aa956cc3-4afd-4a54-b451-3c258cfd139b}
11-28 17:03:48.950  3340  3357 D GeckoViewSettings[C]: onSettingsUpdate {"useMultiprocess":true,"chromeUri":null,"screenId":0,"userAgentOverride":null,"allowJavascript":true,"userAgentMode":0,"useTrackingProtection":false,"suspendMediaWhenInactive":false,"usePrivateMode":false,"displayMode":0,"fullAccessibilityTree":false}
11-28 17:03:48.960  3340  3357 D GeckoViewContent[C]: handleEvent: DOMTitleChanged
11-28 17:03:48.960  3340  3357 D GeckoViewContent[C]: handleEvent: pageshow
11-28 17:03:48.970  3340  3357 D GeckoViewNavigation[C]: shouldLoadURI http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests
11-28 17:03:48.980  3266  3377 I GeckoConsole: PAC file installed from data: URI
11-28 17:03:49.000  3340  3357 D GeckoViewSelectionAction[C]: handleEvent: visibilitychange
11-28 17:03:49.000  3340  3357 D GeckoViewContent[C]: handleEvent: pagehide
11-28 17:03:49.000  3340  3357 D GeckoViewAutoFill: Clearing auto-fill
11-28 17:03:49.010  3340  3357 D GeckoViewContent[C]: handleEvent: DOMTitleChanged
11-28 17:03:49.010  3266  3281 W GeckoEventDispatcher: No listener for GeckoView:OnVisited
11-28 17:03:49.020  3266  3281 D GeckoViewNavigation: onLocationChange
11-28 17:04:03.060  3340  3357 W Web Content: [JavaScript Warning: "Loading failed for the <script> with source “http://mochi.test:8888/tests/SimpleTest/LogController.jsâ€." {file: "http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests" line: 4}]
11-28 17:04:03.070  3340  3357 W Web Content: [JavaScript Warning: "Loading failed for the <script> with source “http://mochi.test:8888/tests/SimpleTest/MemoryStats.jsâ€." {file: "http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests" line: 5}]
11-28 17:04:13.350  3340  3357 W Web Content: [JavaScript Warning: "Loading failed for the <script> with source “http://mochi.test:8888/tests/SimpleTest/TestRunner.jsâ€." {file: "http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests" line: 6}]
11-28 17:04:14.410  3340  3357 E Web Content: [JavaScript Error: "ReferenceError: TestRunner is not defined" {file: "http://mochi.test:8888/tests/SimpleTest/setup.js" line: 9}]
11-28 17:04:14.410  3340  3357 E Web Content: @http://mochi.test:8888/tests/SimpleTest/setup.js:9:1
11-28 17:04:14.420  3340  3357 D GeckoViewNavigation[C]: shouldLoadURI about:blank
11-28 17:04:14.420  3340  3357 D GeckoViewContent[C]: handleEvent: DOMTitleChanged
11-28 17:04:14.440  3266  3281 W GeckoEventDispatcher: No listener for GeckoView:OnVisited
11-28 17:04:14.440  3340  3357 D GeckoViewContent[C]: handleEvent: pageshow
11-28 17:04:14.440  3340  3357 E Web Content: [JavaScript Error: "TypeError: params is undefined; can't access its "manifestFile" property" {file: "http://mochi.test:8888/tests/SimpleTest/setup.js" line: 252}]
11-28 17:04:14.440  3340  3357 E Web Content: hookup@http://mochi.test:8888/tests/SimpleTest/setup.js:252:3
11-28 17:04:14.440  3340  3357 D GeckoViewContent[C]: handleEvent: pageshow
11-28 17:04:14.720  3266  3281 W GeckoEventDispatcher: No listener for GeckoView:GetVisited
11-28 17:07:18.870  1329  1342 I GnssLocationProvider: WakeLock acquired by handleInjectNtpTime()

Comment 7

[Intermittent Failures Robot]

15 failures in 510 pushes (0.029 failures/push) were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 2
* autoland: 13

Platform breakdown:
* android-em-4-3-armv7-api16: 9
* android-hw-p2-8-0-arm7-api-16: 1
* android-em-7-0-x86: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-11-30&endday=2018-11-30&tree=all

Comment 8

[Intermittent Failures Robot]

23 failures in 2808 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 4
* autoland: 17
* mozilla-central: 1
* try: 1

Platform breakdown:
* android-em-7-0-x86: 12
* android-hw-p2-8-0-arm7-api-16: 1
* android-em-4-3-armv7-api16: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-11-26&endday=2018-12-02&tree=all

Comment 9

[Cosmin Sabou [:CosminS]]

There are 22 failures in the last 7 days, all on Android platforms, 11 on android-em-4-3-armv7-api16 debug and opt, 10 on android-em-7-0-x86 opt and 1 on android-hw-p2-8-0-arm7-api-16: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2018-11-30&endday=2018-12-07&tree=all&bug=1500977

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=215618596&repo=autoland&lineNumber=1703

[task 2018-12-05T20:10:59.876Z] 20:10:59     INFO -  53 INFO SimpleTest FINISHED
[task 2018-12-05T20:15:50.226Z] 20:15:50     INFO -  wait for org.mozilla.fennec_aurora complete; top activity=com.android.launcher
[task 2018-12-05T20:15:50.636Z] 20:15:50     INFO -  remoteautomation.py | Application ran for: 0:23:56.827939
[task 2018-12-05T20:15:51.176Z] 20:15:51     INFO -  /data/tombstones does not exist; tombstone check skipped
[task 2018-12-05T20:15:53.014Z] 20:15:53     INFO -  Stopping web server
[task 2018-12-05T20:15:53.045Z] 20:15:53     INFO -  Stopping web socket server
[task 2018-12-05T20:15:53.076Z] 20:15:53     INFO -  Stopping ssltunnel
[task 2018-12-05T20:15:53.118Z] 20:15:53     INFO -  leakcheck | refcount logging is off, so leaks can't be detected!
[task 2018-12-05T20:15:53.118Z] 20:15:53     INFO -  runtests.py | Running tests: end.
[task 2018-12-05T20:15:54.304Z] 20:15:54     INFO -  Buffered messages finished
[task 2018-12-05T20:15:54.305Z] 20:15:54     INFO -  Running manifest: layout/inspector/tests/mochitest.ini
[task 2018-12-05T20:16:00.401Z] 20:16:00     INFO -  adb Ignoring attempt to chmod external storage
[task 2018-12-05T20:16:00.501Z] 20:16:00     INFO -  runtests.py | Failed to copy /builds/worker/workspace/build/tests/mochitest/hyphenation to profile
[task 2018-12-05T20:16:00.933Z] 20:16:00     INFO -  pk12util: PKCS12 IMPORT SUCCESSFUL
[task 2018-12-05T20:16:01.009Z] 20:16:01     INFO -  MochitestServer : launching [u'/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/xpcshell', '-g', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64', '-f', '/builds/worker/workspace/build/hostutils/host-utils-61.0a1.en-US.linux-x86_64/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpPD8mwL.mozrunner'; const _SERVER_PORT = '8854'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/builds/worker/workspace/build/tests/mochitest/server.js']
[task 2018-12-05T20:16:01.009Z] 20:16:01     INFO -  runtests.py | Server pid: 2497
[task 2018-12-05T20:16:01.021Z] 20:16:01     INFO -  runtests.py | Websocket server pid: 2500
[task 2018-12-05T20:16:01.040Z] 20:16:01     INFO -  runtests.py | SSL tunnel pid: 2503
[task 2018-12-05T20:16:15.266Z] 20:16:15     INFO -  adb Ignoring attempt to chmod external storage
[task 2018-12-05T20:16:15.267Z] 20:16:15     INFO -  runtests.py | Running with e10s: False
[task 2018-12-05T20:16:15.268Z] 20:16:15     INFO -  runtests.py | Running with serviceworker_e10s: False
[task 2018-12-05T20:16:15.269Z] 20:16:15     INFO -  runtests.py | Running tests: start.
[task 2018-12-05T20:16:15.901Z] 20:16:15     INFO -  adb launch_application: am start -W -n org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp -a android.intent.action.VIEW --es env9 MOZ_UPLOAD_DIR=/sdcard/tests/mozlog --es env8 R_LOG_DESTINATION=stderr --es args "-no-remote -profile /sdcard/tests/profile//" --es env3 DISABLE_UNSAFE_CPOW_WARNINGS=1 --es env2 R_LOG_VERBOSE=1 --es env1 XPCOM_DEBUG_BREAK=stack --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_LOG_FILE=/sdcard/tests/mozlog/moz.log --es env6 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env5 MOZ_IN_AUTOMATION=1 --es env4 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env12 MOZ_HIDE_RESULTS_TABLE=1 --es env11 R_LOG_LEVEL=6 --es env10 MOZ_CRASHREPORTER_NO_REPORT=1 -d "http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests"
[task 2018-12-05T20:16:33.101Z] 20:16:33     INFO -  remoteautomation.py | Application pid: 1974
[task 2018-12-05T20:24:36.788Z] 20:24:36     INFO -  wait for org.mozilla.fennec_aurora complete; top activity=org.mozilla.fennec_aurora
[task 2018-12-05T20:24:37.800Z] 20:24:37     INFO -  org.mozilla.fennec_aurora unexpectedly found running. Killing...
[task 2018-12-05T20:24:37.801Z] 20:24:37     INFO -  TEST-INFO | started process screentopng
[task 2018-12-05T20:24:38.238Z] 20:24:38     INFO -  TEST-INFO | screentopng: exit 0
[task 2018-12-05T20:24:54.262Z] 20:24:54  WARNING -  TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output
[task 2018-12-05T20:24:54.263Z] 20:24:54     INFO -  remoteautomation.py | Application ran for: 0:08:38.994675
[task 2018-12-05T20:24:54.799Z] 20:24:54     INFO -  /data/tombstones does not exist; tombstone check skipped
[task 2018-12-05T20:24:55.833Z] 20:24:55     INFO -  mozcrash Copy/paste: /usr/local/bin/linux64-minidump_stackwalk /tmp/tmprTmYHv/0c7c0b5e-5300-8ccc-ef9d-106389ee240b.dmp /builds/worker/workspace/build/symbols
[task 2018-12-05T20:25:11.848Z] 20:25:11     INFO -  mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/0c7c0b5e-5300-8ccc-ef9d-106389ee240b.dmp
[task 2018-12-05T20:25:11.849Z] 20:25:11     INFO -  mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/0c7c0b5e-5300-8ccc-ef9d-106389ee240b.extra

Comment 10

[Geoff Brown [:gbrown]]

(In reply to Cosmin Sabou [:CosminS] from comment #9)
> There are 22 failures in the last 7 days, all on Android platforms, 11 on
> android-em-4-3-armv7-api16 debug and opt, 10 on android-em-7-0-x86 opt and 1
> on android-hw-p2-8-0-arm7-api-16:

Many of those are on https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=8dfebfa481b127c422a703354d3b96d0d5d56ee2, which required follow-up patches to land.

However, there are additional failures which should be investigated.

Comment 11

[Intermittent Failures Robot]

4 failures in 1862 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* autoland: 3

Platform breakdown:
* android-em-7-0-x86: 2
* android-em-4-3-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-12-03&endday=2018-12-09&tree=all

Comment 12

[Intermittent Failures Robot]

15 failures in 2485 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 13
* mozilla-inbound: 2

Platform breakdown:
* android-hw-p2-8-0-arm7-api-16: 6
* android-em-4-3-armv7-api16: 9

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-12-10&endday=2018-12-16&tree=all

Comment 13

[Intermittent Failures Robot]

5 failures in 2648 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* mozilla-beta: 1
* try: 1
* mozilla-inbound: 2

Platform breakdown:
* android-em-4-3-armv7-api16: 3
* android-em-7-0-x86: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-12-17&endday=2018-12-23&tree=all

Comment 14

[Intermittent Failures Robot]

2 failures in 1008 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1
* mozilla-central: 1

Platform breakdown:
* android-em-7-0-x86: 1
* android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-12-24&endday=2018-12-30&tree=all

Comment 15

[Intermittent Failures Robot]

4 failures in 1668 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 1
* try: 1
* autoland: 2

Platform breakdown:
* android-em-4-3-armv7-api16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2018-12-31&endday=2019-01-06&tree=all

Comment 16

[Intermittent Failures Robot]

2 failures in 2627 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• try: 1

Platform breakdown:

• android-em-7-0-x86: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-01-07&endday=2019-01-13&tree=all

Comment 17

[Intermittent Failures Robot]

8 failures in 2378 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 5
• mozilla-inbound: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 4
• android-em-7-0-x86: 1
• android-em-4-3-armv7-api16: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-01-14&endday=2019-01-20&tree=all

Comment 18

[Intermittent Failures Robot]

8 failures in 2383 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2
• mozilla-inbound: 3
• autoland: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 5
• android-em-7-0-x86: 2
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-01-21&endday=2019-01-27&tree=all

Comment 19

[Intermittent Failures Robot]

7 failures in 2628 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 4
• mozilla-central: 2
• mozilla-inbound: 1

Platform breakdown:

• android-em-4-3-armv7-api16: 3
• android-hw-p2-8-0-arm7-api-16: 1
• android-em-7-0-x86: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 20

[Intermittent Failures Robot]

15 failures in 2658 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 10
• mozilla-inbound: 5

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 2
• android-em-7-0-x86: 1
• android-em-7-0-x86_64: 2
• android-em-4-3-armv7-api16: 4
• Android-7_0-x86_64: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-02-04&endday=2019-02-10&tree=all

Comment 21

[Intermittent Failures Robot]

15 failures in 2832 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 9
• mozilla-central: 1
• mozilla-beta: 1
• mozilla-inbound: 4

Platform breakdown:

• android-em-4-3-armv7-api16: 2
• android-em-7-0-x86_64: 11
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-02-11&endday=2019-02-17&tree=all

Comment 22

[Intermittent Failures Robot]

4 failures in 2602 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-inbound: 2
• autoland: 2

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 1
• android-em-7-0-x86_64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-02-18&endday=2019-02-24&tree=all

Comment 23

[Intermittent Failures Robot]

24 failures in 537 pushes (0.045 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 24

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 23
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-02-27&endday=2019-02-27&tree=all

Comment 24

[Stefan Hindli [:stefan_hindli]]

Over the last 7 days this bug has 38 failures. These are happening on android-hw-p2-8-0-arm7-api-16, android-em-7-0-x86_64, android-em-4-3-armv7-api16

This is the latest failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=231050849&repo=autoland&lineNumber=3612

Comment 25

[Andrew Halberstadt [:ahal]]

I'm not really involved in Android and looks like Geoff has done some previous investigation here.

Comment 26

[Geoff Brown [:gbrown]]

Most are android-hw (mochitest-media) -- maybe :bc will have a look?

Comment 27

[Bob Clary [:bc] (inactive)]

The most recent spike was from bug 1335740 which landed and was backed out and never merged.

https://treeherder.mozilla.org/#/jobs?repo=autoland&tier=1%2C2%2C3&searchStr=android-hw&fromchange=a1d16d17e6eb59812393c5e049e81f5d765cd3ec&tochange=6b3a949f27efed0c003b79fb1ec97861542c1053

The outstanding intermittent still exists though probably.

Comment 28

[Intermittent Failures Robot]

15 failures in 504 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• try: 1
• mozilla-inbound: 5

Platform breakdown:

• android-em-7-0-x86_64: 14
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-02-28&endday=2019-02-28&tree=all

Comment 29

[Intermittent Failures Robot]

22 failures in 608 pushes (0.036 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 17
• mozilla-inbound: 3
• mozilla-central: 2

Platform breakdown:

• android-em-7-0-x86_64: 21
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-01&endday=2019-03-01&tree=all

Comment 30

[Intermittent Failures Robot]

71 failures in 2981 pushes (0.024 failures/push) were associated with this bug in the last 7 days.

This is the #18 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 56
• try: 2
• mozilla-inbound: 11
• mozilla-central: 2

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 27
• android-em-4-3-armv7-api16: 3
• android-em-7-0-x86_64: 41

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-02-25&endday=2019-03-03&tree=all

Comment 31

[Stefan Hindli [:stefan_hindli]]

Over the last 7 days there are 70 crashes on this bug. These happen on android-hw-p2-8-0-arm7-api-16, android-em-7-0-x86_64, android-em-4-3-armv7-api16

Here is the most recent log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=232585031&repo=autoland&lineNumber=2652

Comment 32

[Geoff Brown [:gbrown]]

Now most are Android x86_64 mochitests, which are tier 2. :snorp added these and is aware of problems.

Comment 33

[Andrew Halberstadt [:ahal]]

Thanks Geoff, /cc snorp

Comment 34

[Intermittent Failures Robot]

60 failures in 2818 pushes (0.021 failures/push) were associated with this bug in the last 7 days.

This is the #25 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 6
• autoland: 47
• mozilla-central: 5
• try: 2

Platform breakdown:

• android-em-7-0-x86_64: 55
• android-hw-p2-8-0-arm7-api-16: 2
• android-em-4-3-armv7-api16: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 35

[Intermittent Failures Robot]

16 failures in 524 pushes (0.031 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 12
• mozilla-central: 2
• mozilla-inbound: 2

Platform breakdown:

• android-em-7-0-x86_64: 14
• android-em-4-3-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-14&endday=2019-03-14&tree=all

Comment 36

[Bob Clary [:bc] (inactive)]

(In reply to Bob Clary [:bc:] from comment #27)

The most recent spike was from bug 1335740 which landed and was backed out and never merged.

https://treeherder.mozilla.org/#/jobs?repo=autoland&tier=1%2C2%2C3&searchStr=android-hw&fromchange=a1d16d17e6eb59812393c5e049e81f5d765cd3ec&tochange=6b3a949f27efed0c003b79fb1ec97861542c1053

The outstanding intermittent still exists though probably.

Whatever effect bug 1335740 had and whatever its backout status is, this never recovered from the increase that occurred around 2019-02-27 https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-02-25&endday=2019-02-27&tree=all&bug=1500977

NI myself to follow up.

Comment 37

[Geoff Brown [:gbrown]]

Most are android-em 7.0 mochitests, which started running in https://hg.mozilla.org/integration/autoland/rev/51efc5e08cee. I am making some progress understanding those in bug 1534732.

Comment 38

[Andreea Pavel [:apavel]]

Removing the disable recommended tag based on the above comments.

Comment 39

[Intermittent Failures Robot]

63 failures in 2771 pushes (0.023 failures/push) were associated with this bug in the last 7 days.

This is the #26 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 42
• mozilla-beta: 2
• mozilla-inbound: 10
• try: 3
• mozilla-central: 6

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 2
• android-em-7-0-x86_64: 55
• android-em-4-3-armv7-api16: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-11&endday=2019-03-17&tree=all

Comment 40

[Intermittent Failures Robot]

18 failures in 512 pushes (0.035 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 15
• mozilla-central: 1
• mozilla-beta: 2

Platform breakdown:

• android-em-7-0-x86_64: 10
• android-hw-p2-8-0-arm7-api-16: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-19&endday=2019-03-19&tree=all

Comment 41

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Bug 1335740 just relanded, causing tier 2 perma-orange with this signature on android. FYI.

Other than the signature, the other details appear to differ from this issue, by being webaudio adjacent rather than a startup issue. But from bug 1335740 comment 19 and 20 there's a chance it might be test harness related (one test was moved to schemed=https which appears to reorder tests)? I thought I'd reach out here if android people with knowledge of the harness are here.

Comment 42

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Geoff were you working on this? I see there was talk of disabling tests in comment 38.

Comment 43

[Geoff Brown [:gbrown]]

I am working on bug 1534732, but I am not working on this and was unaware of the perma-fail. It looks entirely different from 1534732.

Did you see the screenshots?

https://taskcluster-artifacts.net/MXbJhOy0Sc-4EkDsbh-F1w/0/public/test_info/mozilla-test-fail-screenshot_1ZRWPV.png

More generally, the way to debug this is to examine a log:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=234896321&repo=autoland&lineNumber=3111

01:38:57 INFO - adb launch_application: am start -W -n org.mozilla.fennec_aurora/org.mozilla.gecko.BrowserApp -a android.intent.action.VIEW --es env9 MOZ_UPLOAD_DIR=/sdcard/tests/mozlog --es env8 R_LOG_DESTINATION=stderr --es args "-no-remote -profile /sdcard/tests/profile//" --es env3 DISABLE_UNSAFE_CPOW_WARNINGS=1 --es env2 R_LOG_VERBOSE=1 --es env1 XPCOM_DEBUG_BREAK=stack --es env0 MOZ_CRASHREPORTER=1 --es env7 MOZ_LOG_FILE=/sdcard/tests/mozlog/moz.log --es env6 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env5 MOZ_IN_AUTOMATION=1 --es env4 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --es env12 MOZ_HIDE_RESULTS_TABLE=1 --es env11 R_LOG_LEVEL=6 --es env10 MOZ_CRASHREPORTER_NO_REPORT=1 -d "https://example.com:443/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests"
01:38:58 INFO - remoteautomation.py | Application pid: 7131
01:45:28 INFO - wait for org.mozilla.fennec_aurora complete; top activity=org.mozilla.fennec_aurora
01:45:28 INFO - org.mozilla.fennec_aurora unexpectedly found running. Killing...
01:45:29 INFO - TEST-INFO | started process screencap
01:45:29 INFO - TEST-INFO | screencap: exit 0
01:45:40 WARNING - TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output

It looks like fennec was launched at 01:38:57 with pid 7131, then wasn't heard from again; the harness gave up at 01:45:28 and killed it. What was happening on the device? Let's check the logcat for the pid:

https://queue.taskcluster.net/v1/task/MXbJhOy0Sc-4EkDsbh-F1w/runs/0/artifacts/public/test_info/logcat-FA84B1A00013.log

Is this relevant?

03-19 18:39:01.446 7131 7131 D GeckoToolbar: onTabChanged: LOAD_ERROR
03-19 18:39:01.449 7131 7131 D GeckoBrowserApp: BrowserApp.onTabChanged: 1: LOAD_ERROR
03-19 18:39:01.784 7131 7150 W GeckoEventDispatcher: No listener for GeckoView:AccessibilityReady
03-19 18:39:01.835 7131 7131 I GeckoBrowserApp: Found tag
03-19 18:39:01.859 7131 7131 V MediaRouter: Adding route: RouteInfo{ name=Phone, description=null, status=null, category=RouteCategory{ name=System types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
03-19 18:39:01.863 7131 7131 V MediaRouter: Selecting route: RouteInfo{ name=Phone, description=null, status=null, category=RouteCategory{ name=System types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
03-19 18:39:01.865 7131 7131 I MediaRouter: Found default route: MediaRouter.RouteInfo{ uniqueId=android/.support.v7.media.SystemMediaRouteProvider:DEFAULT_ROUTE, name=Phone, description=null, iconUri=null, enabled=true, connecting=false, connectionState=0, canDisconnect=false, playbackType=0, playbackStream=3, deviceType=0, volumeHandling=1, volume=8, volumeMax=25, presentationDisplayId=-1, extras=null, settingsIntent=null, providerPackageName=android }
03-19 18:39:01.865 7131 7131 I MediaRouter: Unselecting the current route because it is no longer selectable: null
03-19 18:39:01.868 7131 7131 D PassiveStumbler: Stopping StumblerService | isDebug:false
03-19 18:39:01.870 7131 7131 V MediaRouter: Selecting route: RouteInfo{ name=Phone, description=null, status=null, category=RouteCategory{ name=System types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
03-19 18:39:01.875 7131 7150 I Gecko : 1553045941875 Marionette TRACE Received observer notification marionette-startup-requested
03-19 18:39:01.876 7131 7150 I Gecko : 1553045941876 Marionette DEBUG Init aborted (running=false, enabled=false, finalUIStartup=true)
03-19 18:39:01.877 1194 2698 W ActivityManager: Permission Denial: Accessing service ComponentInfo{com.google.android.music/com.google.android.music.dial.DialMediaRouteProviderService} from pid=7131, uid=10128 that is not exported from uid 10112
03-19 18:39:01.883 2468 2468 W CastMediaRouteProvider: Not a Cast compatible category: android.media.intent.category.LIVE_VIDEO
03-19 18:39:01.885 2468 6184 I DiscoveryManager: Filter criteria(4574A331,CC1AD845,08FF1091) scannerFlags(2)
03-19 18:39:01.899 2468 6184 E DeviceScanner: [BLE] Failed to start scan because the Bluetooth is disabled.
03-19 18:39:01.904 2468 5902 I qtaguid : Untagging socket 90 failed errno=-22
03-19 18:39:01.904 2468 5902 W NetworkManagementSocketTagger: untagSocket(90) failed with errno -22

Comment 44

[Intermittent Failures Robot]

404 failures in 391 pushes (1.033 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-inbound: 19
• autoland: 367
• mozilla-central: 12
• try: 6

Platform breakdown:

• android-em-7-0-x86_64: 3
• android-hw-p2-8-0-arm7-api-16: 154
• android-hw-p2-8-0-arm7-api-16-pgo: 246
• osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-20&endday=2019-03-20&tree=all

Comment 45

[Intermittent Failures Robot]

209 failures in 475 pushes (0.44 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 17
• autoland: 160
• mozilla-inbound: 32

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 91
• android-hw-p2-8-0-arm7-api-16-pgo: 115
• android-em-7-0-x86_64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-21&endday=2019-03-21&tree=all

Comment 46

[Andreea Pavel [:apavel]]

Recent failures here are https://bugzilla.mozilla.org/show_bug.cgi?id=1335740#c18 and a new bug was filed for those, however the classifications were not modified.

Bug 1537745 has been filed.

Comment 47

[Intermittent Failures Robot]

22 failures in 537 pushes (0.041 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-inbound: 2
• autoland: 11
• mozilla-central: 9

Platform breakdown:

• android-em-7-0-x86_64: 5
• android-hw-p2-8-0-arm7-api-16: 4
• android-hw-p2-8-0-arm7-api-16-pgo: 13

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-22&endday=2019-03-22&tree=all

Comment 48

[Intermittent Failures Robot]

664 failures in 2561 pushes (0.259 failures/push) were associated with this bug in the last 7 days.

This is the #2 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 562
• mozilla-central: 41
• mozilla-beta: 2
• mozilla-inbound: 53
• try: 6

Platform breakdown:

• android-em-7-0-x86_64: 28
• android-em-4-3-armv7-api16: 1
• android-hw-p2-8-0-arm7-api-16: 257
• android-hw-p2-8-0-arm7-api-16-pgo: 377
• osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-18&endday=2019-03-24&tree=all

Comment 49

[Andreea Pavel [:apavel]]

There are 32 total failures in the last 7 days on android-em-7-0-x86_64 opt

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=237053790&repo=autoland&lineNumber=3992

[task 2019-03-30T00:45:06.864Z] 00:45:06 INFO - 759 INFO TEST-START | editor/libeditor/tests/browserscope/test_richtext.html
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 760 INFO TEST-OK | editor/libeditor/tests/browserscope/test_richtext.html | took 1480ms
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 761 INFO TEST-START | Shutdown
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 762 INFO Passed: 136
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 763 INFO Failed: 0
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 764 INFO Todo: 17
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 765 INFO Mode: e10s
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 766 INFO Slowest: 1480ms - /tests/editor/libeditor/tests/browserscope/test_richtext.html
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - 767 INFO SimpleTest FINISHED
[task 2019-03-30T00:45:06.867Z] 00:45:06 INFO - wait for org.mozilla.geckoview.test complete; top activity=com.android.launcher3
[task 2019-03-30T00:45:06.971Z] 00:45:06 INFO - remoteautomation.py | Application ran for: 0:00:13.063371
[task 2019-03-30T00:45:07.507Z] 00:45:07 INFO - Stopping web server
[task 2019-03-30T00:45:07.514Z] 00:45:07 INFO - Stopping web socket server
[task 2019-03-30T00:45:07.534Z] 00:45:07 INFO - Stopping ssltunnel
[task 2019-03-30T00:45:07.554Z] 00:45:07 INFO - leakcheck | refcount logging is off, so leaks can't be detected!
[task 2019-03-30T00:45:07.554Z] 00:45:07 INFO - runtests.py | Running tests: end.
[task 2019-03-30T00:45:08.180Z] 00:45:08 INFO - Buffered messages finished
[task 2019-03-30T00:45:08.180Z] 00:45:08 INFO - Running manifest: editor/libeditor/tests/mochitest.ini
[task 2019-03-30T00:45:08.394Z] 00:45:08 INFO - adb Ignoring attempt to chmod external storage
[task 2019-03-30T00:45:08.408Z] 00:45:08 INFO - runtests.py | Failed to copy /builds/worker/workspace/build/tests/mochitest/hyphenation to profile
[task 2019-03-30T00:45:08.472Z] 00:45:08 INFO - pk12util: PKCS12 IMPORT SUCCESSFUL
[task 2019-03-30T00:45:08.590Z] 00:45:08 INFO - MochitestServer : launching [u'/builds/worker/workspace/build/hostutils/host-utils-67.0a1.en-US.linux-x86_64/xpcshell', '-g', '/builds/worker/workspace/build/hostutils/host-utils-67.0a1.en-US.linux-x86_64', '-f', '/builds/worker/workspace/build/hostutils/host-utils-67.0a1.en-US.linux-x86_64/components/httpd.js', '-e', "const _PROFILE_PATH = '/tmp/tmpxLjvVJ.mozrunner'; const _SERVER_PORT = '8854'; const _SERVER_ADDR = '10.0.2.2'; const _TEST_PREFIX = undefined; const _DISPLAY_RESULTS = false;", '-f', '/builds/worker/workspace/build/tests/mochitest/server.js']
[task 2019-03-30T00:45:08.590Z] 00:45:08 INFO - runtests.py | Server pid: 5129
[task 2019-03-30T00:45:08.592Z] 00:45:08 INFO - runtests.py | Websocket server pid: 5132
[task 2019-03-30T00:45:08.597Z] 00:45:08 INFO - runtests.py | SSL tunnel pid: 5135
[task 2019-03-30T00:45:08.874Z] 00:45:08 INFO - adb Ignoring attempt to chmod external storage
[task 2019-03-30T00:45:08.874Z] 00:45:08 INFO - runtests.py | Running with scheme: http
[task 2019-03-30T00:45:08.874Z] 00:45:08 INFO - runtests.py | Running with e10s: True
[task 2019-03-30T00:45:08.874Z] 00:45:08 INFO - runtests.py | Running with serviceworker_e10s: False
[task 2019-03-30T00:45:08.874Z] 00:45:08 INFO - runtests.py | Running with socketprocess_e10s: False
[task 2019-03-30T00:45:08.874Z] 00:45:08 INFO - runtests.py | Running tests: start.
[task 2019-03-30T00:45:09.184Z] 00:45:09 INFO - adb Granting important runtime permissions to org.mozilla.geckoview.test
[task 2019-03-30T00:45:10.502Z] 00:45:10 INFO - adb launch_application: am start -W -n org.mozilla.geckoview.test/org.mozilla.geckoview.test.TestRunnerActivity -a android.intent.action.Main --es env9 MOZ_CRASHREPORTER_NO_REPORT=1 --es env8 MOZ_UPLOAD_DIR=/sdcard/tests/mozlog --es args "-no-remote -profile /sdcard/tests/profile//" --es env3 DISABLE_UNSAFE_CPOW_WARNINGS=1 --es env2 R_LOG_VERBOSE=1 --es env1 XPCOM_DEBUG_BREAK=stack --es env0 MOZ_CRASHREPORTER=1 --es env7 R_LOG_DESTINATION=stderr --es env6 MOZ_CRASHREPORTER_SHUTDOWN=1 --es env5 MOZ_IN_AUTOMATION=1 --es env4 MOZ_DISABLE_NONLOCAL_CONNECTIONS=1 --ez use_multiprocess True --es env11 MOZ_HIDE_RESULTS_TABLE=1 --es env10 R_LOG_LEVEL=6 -d "http://mochi.test:8888/tests?autorun=1&closeWhenDone=1&logFile=%2Fsdcard%2Ftests%2Flogs%2Fmochitest.log&fileLevel=INFO&consoleLevel=INFO&hideResultsTable=1&manifestFile=tests.json&dumpOutputDirectory=%2Fsdcard%2Ftests"
[task 2019-03-30T00:45:11.610Z] 00:45:11 INFO - remoteautomation.py | Application pid: 8387
[task 2019-03-30T00:51:39.754Z] 00:51:39 INFO - wait for org.mozilla.geckoview.test complete; top activity=org.mozilla.geckoview.test
[task 2019-03-30T00:51:39.858Z] 00:51:39 INFO - org.mozilla.geckoview.test unexpectedly found running. Killing...
[task 2019-03-30T00:51:39.859Z] 00:51:39 INFO - TEST-INFO | started process screentopng
[task 2019-03-30T00:51:40.183Z] 00:51:40 INFO - TEST-INFO | screentopng: exit 0
[task 2019-03-30T00:51:54.533Z] 00:51:54 WARNING - TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output
[task 2019-03-30T00:51:54.533Z] 00:51:54 INFO - remoteautomation.py | Application ran for: 0:06:45.555810
[task 2019-03-30T00:51:54.971Z] 00:51:54 INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/CjI19ru1QGm8pm3v7bBYBA/artifacts/public/build/target.crashreporter-symbols.zip
[task 2019-03-30T00:52:17.065Z] 00:52:17 INFO - mozcrash Copy/paste: /builds/worker/workspace/build/linux64-minidump_stackwalk /tmp/tmp0NvrJg/02913814-d4ab-a291-58c8-f5645b84021a.dmp /tmp/tmpw7ZOlI
[task 2019-03-30T00:52:24.675Z] 00:52:24 INFO - mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/02913814-d4ab-a291-58c8-f5645b84021a.dmp
[task 2019-03-30T00:52:24.675Z] 00:52:24 INFO - mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/02913814-d4ab-a291-58c8-f5645b84021a.extra
[task 2019-03-30T00:52:24.678Z] 00:52:24 WARNING - PROCESS-CRASH | remoteautomation.py | application crashed [@ libc.so + 0x8c66a]

Andrew can you assign someone to take a look?

Comment 50

[Intermittent Failures Robot]

39 failures in 3910 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 30
• mozilla-beta: 4
• mozilla-inbound: 2
• mozilla-central: 2
• try: 1

Platform breakdown:

• android-em-7-0-x86_64: 38
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-03-25&endday=2019-03-31&tree=all

Comment 51

[Andrew Halberstadt [:ahal]]

The timeout happens when restarting geckoview between manifests. Right after launching the app we should start to see tests run, but instead there is no output:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=237053790&repo=autoland&lineNumber=3986

This is well outside my area of expertise. Geoff, Bob, you guys have looked at this in the past. Do you have any more insight into what might be happening since the last time you looked? Could the bug be in geckoview / the TestRunnerActivity?

Comment 52

[Geoff Brown [:gbrown]]

For the android-em* cases, there is on-going work in bug 1534732. (That might apply to android-hw* or even desktop failures as well, but I'm investigating android-em since that's easiest to reproduce.) :edenchuang is investigating a similar issue that might have the same cause in bug 1427396.

So far it looks like ScriptLoader is intermittently failing and that might be due to a necko (or httpd?) bug.

Comment 53

[Intermittent Failures Robot]

47 failures in 2843 pushes (0.017 failures/push) were associated with this bug in the last 7 days.

This is the #49 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 35
• mozilla-central: 4
• try: 3
• mozilla-beta: 1
• mozilla-inbound: 4

Platform breakdown:

• android-em-7-0-x86_64: 46
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-04-01&endday=2019-04-07&tree=all

Comment 54

[Bob Clary [:bc] (inactive)]

If we had support to turn debug logging on in server.js:

https://searchfox.org/mozilla-central/source/testing/mochitest/server.js#342

the same way we shut down the server:

https://searchfox.org/mozilla-central/rev/7abb9117c8500ed20833746c9f8e800fce3a4688/testing/mochitest/server.js#330

we could add something like

https://searchfox.org/mozilla-central/source/testing/mochitest/runtests.py#438
https://searchfox.org/mozilla-central/source/testing/mochitest/runtests.py#528

and get debug output from server.js.

Though it is not related to this issue, in the future you can also use:
https://searchfox.org/mozilla-central/source/testing/mochitest/ssltunnel/ssltunnel.cpp#44

for ssltunnel related logging. It would be good to have an easy way to turn this on.

I think the next step is to figure out why these scripts fail to load.

Comment 55

[Intermittent Failures Robot]

43 failures in 3025 pushes (0.014 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 7
• autoland: 32
• mozilla-beta: 1
• mozilla-central: 3

Platform breakdown:

• android-em-7-0-x86_64: 41
• android-em-4-3-armv7-api16: 1
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-04-08&endday=2019-04-14&tree=all

Comment 56

[Andreea Pavel [:apavel]]

There are 46 total failures in the last 7 days on android-em-4-3-armv7-api16 pgo, android-em-7-0-x86_64 opt (majority) and android-hw-p2-8-0-arm7-api-16 pgo.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=240572463&repo=autoland&lineNumber=2671

[task 2019-04-16T07:28:22.659Z] 07:28:22 INFO - 626 INFO TEST-START | dom/jsurl/test/test_bug384981.html
[task 2019-04-16T07:28:22.659Z] 07:28:22 INFO - 627 INFO TEST-OK | dom/jsurl/test/test_bug384981.html | took 60ms
[task 2019-04-16T07:28:22.659Z] 07:28:22 INFO - 628 INFO TEST-START | Shutdown

[task 2019-04-16T07:34:55.815Z] 07:34:55 INFO - org.mozilla.geckoview.test unexpectedly found running. Killing...
[task 2019-04-16T07:34:55.816Z] 07:34:55 INFO - TEST-INFO | started process screentopng
[task 2019-04-16T07:34:56.093Z] 07:34:56 INFO - TEST-INFO | screentopng: exit 0
[task 2019-04-16T07:35:05.332Z] 07:35:05 WARNING - TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output
[task 2019-04-16T07:35:05.332Z] 07:35:05 INFO - remoteautomation.py | Application ran for: 0:06:40.578361
[task 2019-04-16T07:35:05.864Z] 07:35:05 INFO - mozcrash Downloading symbols from: https://queue.taskcluster.net/v1/task/LorD6wvqSpKu0B0WHEN2_w/artifacts/public/build/target.crashreporter-symbols.zip
[task 2019-04-16T07:35:09.340Z] 07:35:09 INFO - mozcrash Copy/paste: /builds/worker/workspace/build/linux64-minidump_stackwalk /tmp/tmpzFhxvB/50deb535-84e0-4f10-a8e5-ec25b4ac1614.dmp /tmp/tmpva63r6
[task 2019-04-16T07:35:12.647Z] 07:35:12 INFO - mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/50deb535-84e0-4f10-a8e5-ec25b4ac1614.dmp
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/50deb535-84e0-4f10-a8e5-ec25b4ac1614.extra
[task 2019-04-16T07:35:12.650Z] 07:35:12 WARNING - PROCESS-CRASH | remoteautomation.py | application crashed [@ libc.so + 0x8c66a]
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - Crash dump filename: /tmp/tmpzFhxvB/50deb535-84e0-4f10-a8e5-ec25b4ac1614.dmp
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - Operating system: Android
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - 0.0.0 Linux 3.10.0+ #1 PREEMPT Thu Jan 5 00:46:30 UTC 2017 x86_64
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - CPU: amd64
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - family 6 model 2 stepping 3
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - 1 CPU
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - GPU: UNKNOWN
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - Crash reason: SIGABRT
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - Crash address: 0x1320
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - Process uptime: not available
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - Thread 0 (crashed)
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - 0 libc.so + 0x8c66a
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - rax = 0xfffffffffffffffc rdx = 0x0000000000000010
[task 2019-04-16T07:35:12.650Z] 07:35:12 INFO - rcx = 0xffffffffffffffff rbx = 0x00000000ffffffff
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - rsi = 0x00007ffff8689880 rdi = 0x000000000000001c
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - rbp = 0x00000000ffffffff rsp = 0x00007ffff8689808
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - r8 = 0x0000000000000000 r9 = 0x0000000000000008
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - r10 = 0x00000000ffffffff r11 = 0x0000000000000246
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - r12 = 0x00000000ffffffff r13 = 0x000076b8ae3e3f40
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - r14 = 0x0000000000000000 r15 = 0x0000000000000000
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - rip = 0x000076b8b0ffe66a
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - Found by: given as instruction pointer in context
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - 1 libc.so + 0x2a0e6
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - rsp = 0x00007ffff8689810 rip = 0x000076b8b0f9c0e6
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - 2 libutils.so + 0x184e7
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - rsp = 0x00007ffff8689830 rip = 0x000076b8b0b084e7
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - 3 system@[e-mail] + 0x3d5350
[task 2019-04-16T07:35:12.651Z] 07:35:12 INFO - rsp = 0x00007ffff8689858 rip = 0x0000000071675350
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - 4 boot-framework.oat + 0x1000
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - rsp = 0x00007ffff8689868 rip = 0x0000000072f54000
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - 5 dalvik-main space (deleted) + 0x32f80
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - rsp = 0x00007ffff8689880 rip = 0x0000000012c32f80
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - 6 libart.so + 0x2d39f9
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - rsp = 0x00007ffff86898a0 rip = 0x000076b8adddf9f9
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - 7 boot-framework.oat + 0x15bc3f0
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - rsp = 0x00007ffff86898a8 rip = 0x000000007450f3f0
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.652Z] 07:35:12 INFO - 8 libart.so + 0x4f1be9
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - rsp = 0x00007ffff86898b0 rip = 0x000076b8adffdbe9
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - 9 system@[e-mail] + 0x2a9f48
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - rsp = 0x00007ffff86898c8 rip = 0x0000000071549f48
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - Found by: stack scanning
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - 10 system@[e-mail] + 0x3d5350
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - rsp = 0x00007ffff86898e0 rip = 0x0000000071675350
[task 2019-04-16T07:35:12.653Z] 07:35:12 INFO - Found by: stack scanning

Comment 57

[Intermittent Failures Robot]

39 failures in 2673 pushes (0.015 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 5
• autoland: 29
• try: 1
• mozilla-central: 4

Platform breakdown:

• android-em-7-0-x86_64: 37
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-04-15&endday=2019-04-21&tree=all

Comment 58

[Razvan Maries]

There are 48 total failures in the last 7 days on android-em-7-0-x86_64 opt.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=242122484&repo=autoland&lineNumber=1502

task 2019-04-23T19:12:16.533Z] 19:12:16 INFO - TEST-START | dom/security/test/mixedcontentblocker/test_main.html
[task 2019-04-23T19:23:10.674Z] 19:23:10 WARNING - TEST-UNEXPECTED-FAIL | remoteautomation.py | application timed out after 370 seconds with no output
[task 2019-04-23T19:23:10.675Z] 19:23:10 INFO - remoteautomation.py | Application ran for: 0:10:45.255944
[task 2019-04-23T19:23:13.952Z] 19:23:13 INFO - mozcrash Copy/paste: /builds/worker/workspace/build/linux64-minidump_stackwalk /tmp/tmp5mRMmq/616d1452-87dc-c083-c4b9-2e716de1ea4d.dmp /builds/worker/workspace/build/symbols
[task 2019-04-23T19:23:33.560Z] 19:23:33 INFO - mozcrash Saved minidump as /builds/worker/workspace/build/blobber_upload_dir/616d1452-87dc-c083-c4b9-2e716de1ea4d.dmp
[task 2019-04-23T19:23:33.564Z] 19:23:33 INFO - mozcrash Saved app info as /builds/worker/workspace/build/blobber_upload_dir/616d1452-87dc-c083-c4b9-2e716de1ea4d.extra
[task 2019-04-23T19:23:33.582Z] 19:23:33 WARNING - PROCESS-CRASH | remoteautomation.py | application crashed [@ libGLESv2_enc.so + 0x9804]
[task 2019-04-23T19:23:33.584Z] 19:23:33 INFO - Crash dump filename: /tmp/tmp5mRMmq/616d1452-87dc-c083-c4b9-2e716de1ea4d.dmp
[task 2019-04-23T19:23:33.585Z] 19:23:33 INFO - Operating system: Android
[task 2019-04-23T19:23:33.586Z] 19:23:33 INFO - 0.0.0 Linux 2.6.29-gea477bb #1 Wed Sep 26 11:04:45 PDT 2012 armv7l
[task 2019-04-23T19:23:33.587Z] 19:23:33 INFO - CPU: arm
[task 2019-04-23T19:23:33.588Z] 19:23:33 INFO - ARMv7 ARM Cortex-A8 features: swp,half,thumb,fastmult,vfpv2,edsp,neon,vfpv3
[task 2019-04-23T19:23:33.589Z] 19:23:33 INFO - 1 CPU
[task 2019-04-23T19:23:33.589Z] 19:23:33 INFO - GPU: UNKNOWN
[task 2019-04-23T19:23:33.590Z] 19:23:33 INFO - Crash reason: SIGABRT
[task 2019-04-23T19:23:33.591Z] 19:23:33 INFO - Crash address: 0x4af
[task 2019-04-23T19:23:33.592Z] 19:23:33 INFO - Process uptime: not available
[task 2019-04-23T19:23:33.593Z] 19:23:33 INFO - Thread 0 (crashed)
[task 2019-04-23T19:23:33.594Z] 19:23:33 INFO - 0 libGLESv2_enc.so + 0x9804
[task 2019-04-23T19:23:33.597Z] 19:23:33 INFO - r0 = 0x2a1fbe88 r1 = 0xbe8962c0 r2 = 0xffffffff r3 = 0xbe8962c0
[task 2019-04-23T19:23:33.600Z] 19:23:33 INFO - r4 = 0x2a1fbe88 r5 = 0xffffffff r6 = 0x00000003 r7 = 0xbe8962fc
[task 2019-04-23T19:23:33.603Z] 19:23:33 INFO - r8 = 0x000084c1 r9 = 0x2a1ef260 r10 = 0xbe896300 r12 = 0x5ced1f20
[task 2019-04-23T19:23:33.605Z] 19:23:33 INFO - fp = 0x43250000 sp = 0xbe8962c0 lr = 0x5cecc59b pc = 0x5cec2804
[task 2019-04-23T19:23:33.606Z] 19:23:33 INFO - Found by: given as instruction pointer in context
[task 2019-04-23T19:23:33.608Z] 19:23:33 INFO - 1 libGLESv2_enc.so + 0x13d87
[task 2019-04-23T19:23:33.609Z] 19:23:33 INFO - sp = 0xbe8962d0 pc = 0x5ceccd89
[task 2019-04-23T19:23:33.610Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.612Z] 19:23:33 INFO - 2 libGLESv2_enc.so + 0xc815
[task 2019-04-23T19:23:33.613Z] 19:23:33 INFO - sp = 0xbe8962f0 pc = 0x5cec5817
[task 2019-04-23T19:23:33.615Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.616Z] 19:23:33 INFO - 3 libutils.so + 0x12b61
[task 2019-04-23T19:23:33.620Z] 19:23:33 INFO - sp = 0xbe8962f8 pc = 0x400afb63
[task 2019-04-23T19:23:33.622Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.624Z] 19:23:33 INFO - 4 dalvik-heap (deleted) + 0x1d3affe
[task 2019-04-23T19:23:33.626Z] 19:23:33 INFO - sp = 0xbe896320 pc = 0x431f0000
[task 2019-04-23T19:23:33.629Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.637Z] 19:23:33 INFO - 5 dalvik-heap (deleted) + 0x2fcaffe
[task 2019-04-23T19:23:33.640Z] 19:23:33 INFO - sp = 0xbe89632c pc = 0x44480000
[task 2019-04-23T19:23:33.645Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.645Z] 19:23:33 INFO - 6 libGLESv2_emulation.so + 0x417d
[task 2019-04-23T19:23:33.646Z] 19:23:33 INFO - sp = 0xbe896330 pc = 0x52dba17f
[task 2019-04-23T19:23:33.649Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.651Z] 19:23:33 INFO - 7 libhwui.so + 0x258b5
[task 2019-04-23T19:23:33.652Z] 19:23:33 INFO - sp = 0xbe896338 pc = 0x40b8a8b7
[task 2019-04-23T19:23:33.652Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.656Z] 19:23:33 INFO - 8 libhwui.so + 0x32636
[task 2019-04-23T19:23:33.658Z] 19:23:33 INFO - sp = 0xbe89633c pc = 0x40b97638
[task 2019-04-23T19:23:33.661Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.663Z] 19:23:33 INFO - 9 libhwui.so + 0x1c88b
[task 2019-04-23T19:23:33.664Z] 19:23:33 INFO - sp = 0xbe896348 pc = 0x40b8188d
[task 2019-04-23T19:23:33.666Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.668Z] 19:23:33 INFO - 10 libhwui.so + 0x1dcaf
[task 2019-04-23T19:23:33.671Z] 19:23:33 INFO - sp = 0xbe896358 pc = 0x40b82cb1
[task 2019-04-23T19:23:33.678Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.682Z] 19:23:33 INFO - 11 libc.so + 0x47ffe
[task 2019-04-23T19:23:33.684Z] 19:23:33 INFO - sp = 0xbe89635c pc = 0x4006f000
[task 2019-04-23T19:23:33.685Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.687Z] 19:23:33 INFO - 12 libhwui.so + 0x1dd25
[task 2019-04-23T19:23:33.689Z] 19:23:33 INFO - sp = 0xbe896390 pc = 0x40b82d27
[task 2019-04-23T19:23:33.690Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.691Z] 19:23:33 INFO - 13 dalvik-heap (deleted) + 0x1d9affe
[task 2019-04-23T19:23:33.693Z] 19:23:33 INFO - sp = 0xbe896394 pc = 0x43250000
[task 2019-04-23T19:23:33.696Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.699Z] 19:23:33 INFO - 14 dalvik-heap (deleted) + 0x1d9affe
[task 2019-04-23T19:23:33.700Z] 19:23:33 INFO - sp = 0xbe8963a4 pc = 0x43250000
[task 2019-04-23T19:23:33.703Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.705Z] 19:23:33 INFO - 15 libhwui.so + 0x1dced
[task 2019-04-23T19:23:33.708Z] 19:23:33 INFO - sp = 0xbe8963a8 pc = 0x40b82cef
[task 2019-04-23T19:23:33.710Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.712Z] 19:23:33 INFO - 16 libhwui.so + 0x13ee7
[task 2019-04-23T19:23:33.718Z] 19:23:33 INFO - sp = 0xbe8963b8 pc = 0x40b78ee9
[task 2019-04-23T19:23:33.719Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.722Z] 19:23:33 INFO - 17 libhwui.so + 0x1374f
[task 2019-04-23T19:23:33.724Z] 19:23:33 INFO - sp = 0xbe8963c0 pc = 0x40b78751
[task 2019-04-23T19:23:33.725Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.726Z] 19:23:33 INFO - 18 libhwui.so + 0x32ab2
[task 2019-04-23T19:23:33.727Z] 19:23:33 INFO - sp = 0xbe8963cc pc = 0x40b97ab4
[task 2019-04-23T19:23:33.730Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.732Z] 19:23:33 INFO - 19 libhwui.so + 0x1367b
[task 2019-04-23T19:23:33.735Z] 19:23:33 INFO - sp = 0xbe8963e0 pc = 0x40b7867d
[task 2019-04-23T19:23:33.738Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.738Z] 19:23:33 INFO - 20 libhwui.so + 0x1bac3
[task 2019-04-23T19:23:33.739Z] 19:23:33 INFO - sp = 0xbe896430 pc = 0x40b80ac5
[task 2019-04-23T19:23:33.741Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.741Z] 19:23:33 INFO - 21 libhwui.so + 0x31e8e
[task 2019-04-23T19:23:33.743Z] 19:23:33 INFO - sp = 0xbe896444 pc = 0x40b96e90
[task 2019-04-23T19:23:33.743Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.744Z] 19:23:33 INFO - 22 libhwui.so + 0x31ed6
[task 2019-04-23T19:23:33.746Z] 19:23:33 INFO - sp = 0xbe89645c pc = 0x40b96ed8
[task 2019-04-23T19:23:33.747Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.749Z] 19:23:33 INFO - 23 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.750Z] 19:23:33 INFO - sp = 0xbe896494 pc = 0x40b96f20
[task 2019-04-23T19:23:33.752Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.754Z] 19:23:33 INFO - 24 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.756Z] 19:23:33 INFO - sp = 0xbe8964b8 pc = 0x40b96f20
[task 2019-04-23T19:23:33.757Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.758Z] 19:23:33 INFO - 25 libdvm.so + 0x386ff
[task 2019-04-23T19:23:33.759Z] 19:23:33 INFO - sp = 0xbe8964c0 pc = 0x40876701
[task 2019-04-23T19:23:33.760Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.762Z] 19:23:33 INFO - 26 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.764Z] 19:23:33 INFO - sp = 0xbe8964dc pc = 0x40b96f20
[task 2019-04-23T19:23:33.768Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.769Z] 19:23:33 INFO - 27 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.770Z] 19:23:33 INFO - sp = 0xbe896500 pc = 0x40b96f20
[task 2019-04-23T19:23:33.774Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.778Z] 19:23:33 INFO - 28 dalvik-heap (deleted) + 0x154afff
[task 2019-04-23T19:23:33.780Z] 19:23:33 INFO - sp = 0xbe896508 pc = 0x42a00001
[task 2019-04-23T19:23:33.787Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.789Z] 19:23:33 INFO - 29 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.790Z] 19:23:33 INFO - sp = 0xbe896524 pc = 0x40b96f20
[task 2019-04-23T19:23:33.792Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.795Z] 19:23:33 INFO - 30 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.808Z] 19:23:33 INFO - sp = 0xbe896548 pc = 0x40b96f20
[task 2019-04-23T19:23:33.809Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.818Z] 19:23:33 INFO - 31 dalvik-heap (deleted) + 0x1d3afff
[task 2019-04-23T19:23:33.819Z] 19:23:33 INFO - sp = 0xbe896550 pc = 0x431f0001
[task 2019-04-23T19:23:33.829Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.830Z] 19:23:33 INFO - 32 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.831Z] 19:23:33 INFO - sp = 0xbe89656c pc = 0x40b96f20
[task 2019-04-23T19:23:33.833Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.834Z] 19:23:33 INFO - 33 dalvik-heap (deleted) + 0x1d9afff
[task 2019-04-23T19:23:33.835Z] 19:23:33 INFO - sp = 0xbe896574 pc = 0x43250001
[task 2019-04-23T19:23:33.836Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.837Z] 19:23:33 INFO - 34 libhwui.so + 0x31f1e
[task 2019-04-23T19:23:33.838Z] 19:23:33 INFO - sp = 0xbe896590 pc = 0x40b96f20
[task 2019-04-23T19:23:33.839Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.840Z] 19:23:33 INFO - 35 libhwui.so + 0x1baeb
[task 2019-04-23T19:23:33.842Z] 19:23:33 INFO - sp = 0xbe8965b8 pc = 0x40b80aed
[task 2019-04-23T19:23:33.844Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.846Z] 19:23:33 INFO - 36 libandroid_runtime.so + 0x64a4b
[task 2019-04-23T19:23:33.851Z] 19:23:33 INFO - sp = 0xbe8965d0 pc = 0x4016aa4d
[task 2019-04-23T19:23:33.853Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.856Z] 19:23:33 INFO - 37 libhwui.so + 0x21521
[task 2019-04-23T19:23:33.858Z] 19:23:33 INFO - sp = 0xbe8965d8 pc = 0x40b86523
[task 2019-04-23T19:23:33.860Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.862Z] 19:23:33 INFO - 38 dalvik-LinearAlloc (deleted) + 0x65f86
[task 2019-04-23T19:23:33.865Z] 19:23:33 INFO - sp = 0xbe8965f4 pc = 0x4c4ccf88
[task 2019-04-23T19:23:33.868Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.872Z] 19:23:33 INFO - 39 libdvm.so + 0x1dc4e
[task 2019-04-23T19:23:33.874Z] 19:23:33 INFO - sp = 0xbe896600 pc = 0x4085bc50
[task 2019-04-23T19:23:33.875Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.877Z] 19:23:33 INFO - 40 framework.odex + 0x5d12ca
[task 2019-04-23T19:23:33.879Z] 19:23:33 INFO - sp = 0xbe896614 pc = 0x4e0a32cc
[task 2019-04-23T19:23:33.884Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.885Z] 19:23:33 INFO - 41 dalvik-heap (deleted) + 0x2a31e
[task 2019-04-23T19:23:33.886Z] 19:23:33 INFO - sp = 0xbe896618 pc = 0x414df320
[task 2019-04-23T19:23:33.887Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.889Z] 19:23:33 INFO - 42 libdvm.so + 0x4dcad
[task 2019-04-23T19:23:33.890Z] 19:23:33 INFO - sp = 0xbe896620 pc = 0x4088bcaf
[task 2019-04-23T19:23:33.892Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.893Z] 19:23:33 INFO - 43 framework.odex + 0x5d12c5
[task 2019-04-23T19:23:33.894Z] 19:23:33 INFO - sp = 0xbe896628 pc = 0x4e0a32c7
[task 2019-04-23T19:23:33.895Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.897Z] 19:23:33 INFO - 44 libandroid_runtime.so + 0x64a2b
[task 2019-04-23T19:23:33.898Z] 19:23:33 INFO - sp = 0xbe89662c pc = 0x4016aa2d
[task 2019-04-23T19:23:33.900Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.901Z] 19:23:33 INFO - 45 libc.so + 0x105f5
[task 2019-04-23T19:23:33.902Z] 19:23:33 INFO - sp = 0xbe896650 pc = 0x400375f7
[task 2019-04-23T19:23:33.904Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.907Z] 19:23:33 INFO - 46 libdvm.so + 0xae6a2
[task 2019-04-23T19:23:33.909Z] 19:23:33 INFO - sp = 0xbe89665c pc = 0x408ec6a4
[task 2019-04-23T19:23:33.912Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.917Z] 19:23:33 INFO - 47 libdvm.so + 0x728bb
[task 2019-04-23T19:23:33.919Z] 19:23:33 INFO - sp = 0xbe896678 pc = 0x408b08bd
[task 2019-04-23T19:23:33.922Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.923Z] 19:23:33 INFO - 48 libdvm.so + 0xa9c86
[task 2019-04-23T19:23:33.925Z] 19:23:33 INFO - sp = 0xbe89668c pc = 0x408e7c88
[task 2019-04-23T19:23:33.928Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.931Z] 19:23:33 INFO - 49 libdvm.so + 0x7845b
[task 2019-04-23T19:23:33.939Z] 19:23:33 INFO - sp = 0xbe896690 pc = 0x408b645d
[task 2019-04-23T19:23:33.942Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.944Z] 19:23:33 INFO - 50 libdvm.so + 0xa9c86
[task 2019-04-23T19:23:33.946Z] 19:23:33 INFO - sp = 0xbe89669c pc = 0x408e7c88
[task 2019-04-23T19:23:33.948Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.949Z] 19:23:33 INFO - 51 dalvik-LinearAlloc (deleted) + 0x1126
[task 2019-04-23T19:23:33.952Z] 19:23:33 INFO - sp = 0xbe8966a0 pc = 0x4c468128
[task 2019-04-23T19:23:33.953Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.955Z] 19:23:33 INFO - 52 dalvik-heap (deleted) + 0x2ac006
[task 2019-04-23T19:23:33.957Z] 19:23:33 INFO - sp = 0xbe8966b8 pc = 0x41761008
[task 2019-04-23T19:23:33.962Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.963Z] 19:23:33 INFO - 53 dalvik-heap (deleted) + 0x6afc0e
[task 2019-04-23T19:23:33.965Z] 19:23:33 INFO - sp = 0xbe8966bc pc = 0x41b64c10
[task 2019-04-23T19:23:33.966Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.968Z] 19:23:33 INFO - 54 dalvik-heap (deleted) + 0x6afc0e
[task 2019-04-23T19:23:33.969Z] 19:23:33 INFO - sp = 0xbe8966c4 pc = 0x41b64c10
[task 2019-04-23T19:23:33.972Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.974Z] 19:23:33 INFO - 55 dalvik-heap (deleted) + 0x2ac006
[task 2019-04-23T19:23:33.977Z] 19:23:33 INFO - sp = 0xbe8966c8 pc = 0x41761008
[task 2019-04-23T19:23:33.980Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.982Z] 19:23:33 INFO - 56 dalvik-heap (deleted) + 0x2ac006
[task 2019-04-23T19:23:33.983Z] 19:23:33 INFO - sp = 0xbe8966e8 pc = 0x41761008
[task 2019-04-23T19:23:33.985Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.985Z] 19:23:33 INFO - 57 dalvik-heap (deleted) + 0x6afcc6
[task 2019-04-23T19:23:33.986Z] 19:23:33 INFO - sp = 0xbe8966ec pc = 0x41b64cc8
[task 2019-04-23T19:23:33.987Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.988Z] 19:23:33 INFO - 58 dalvik-heap (deleted) + 0x6afcc6
[task 2019-04-23T19:23:33.990Z] 19:23:33 INFO - sp = 0xbe8966f4 pc = 0x41b64cc8
[task 2019-04-23T19:23:33.991Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.993Z] 19:23:33 INFO - 59 libc.so + 0x3b072
[task 2019-04-23T19:23:33.996Z] 19:23:33 INFO - sp = 0xbe896700 pc = 0x40062074
[task 2019-04-23T19:23:33.997Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:33.999Z] 19:23:33 INFO - 60 dalvik-LinearAlloc (deleted) + 0x65f86
[task 2019-04-23T19:23:34.004Z] 19:23:33 INFO - sp = 0xbe896718 pc = 0x4c4ccf88
[task 2019-04-23T19:23:34.007Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.009Z] 19:23:33 INFO - 61 dalvik-heap (deleted) + 0x2a31e
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - sp = 0xbe896724 pc = 0x414df320
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - 62 libdvm.so + 0x1ddbe
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - sp = 0xbe896728 pc = 0x4085bdc0
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - 63 libdvm.so + 0x385e3
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - sp = 0xbe896738 pc = 0x408765e5
[task 2019-04-23T19:23:34.010Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - 64 dalvik-LinearAlloc (deleted) + 0x65f86
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - sp = 0xbe896744 pc = 0x4c4ccf88
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - 65 framework.odex + 0x40c316
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - sp = 0xbe896748 pc = 0x4dede318
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - 66 dalvik-heap (deleted) + 0x2a31e
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - sp = 0xbe896754 pc = 0x414df320
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.011Z] 19:23:33 INFO - 67 libdvm.so + 0x27062
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - sp = 0xbe896758 pc = 0x40865064
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - 68 libdvm.so + 0x2de9a
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - sp = 0xbe896764 pc = 0x4086be9c
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - 69 dalvik-LinearAlloc (deleted) + 0xa79b6
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - sp = 0xbe896778 pc = 0x4c50e9b8
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - 70 dalvik-heap (deleted) + 0x2b75ea
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - sp = 0xbe89677c pc = 0x4176c5ec
[task 2019-04-23T19:23:34.012Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.013Z] 19:23:33 INFO - 71 libdvm.so + 0x2b582
[task 2019-04-23T19:23:34.015Z] 19:23:33 INFO - sp = 0xbe896780 pc = 0x40869584
[task 2019-04-23T19:23:34.015Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.015Z] 19:23:33 INFO - 72 dalvik-heap (deleted) + 0x179fe6
[task 2019-04-23T19:23:34.015Z] 19:23:33 INFO - sp = 0xbe89678c pc = 0x4162efe8
[task 2019-04-23T19:23:34.015Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.015Z] 19:23:33 INFO - 73 framework.odex + 0x50c326
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - sp = 0xbe896790 pc = 0x4dfde328
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - 74 dalvik-LinearAlloc (deleted) + 0xc2a6
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - sp = 0xbe896798 pc = 0x4c4732a8
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - 75 core.odex + 0x364ffe
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - sp = 0xbe89679c pc = 0x4d7cd000
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - 76 dalvik-heap (deleted) + 0x2b7556
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - sp = 0xbe8967a0 pc = 0x4176c558
[task 2019-04-23T19:23:34.016Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.017Z] 19:23:33 INFO - 77 dalvik-LinearAlloc (deleted) + 0xa79b6
[task 2019-04-23T19:23:34.017Z] 19:23:33 INFO - sp = 0xbe8967f8 pc = 0x4c50e9b8
[task 2019-04-23T19:23:34.019Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.019Z] 19:23:33 INFO - 78 libdvm.so + 0xa9c86
[task 2019-04-23T19:23:34.019Z] 19:23:33 INFO - sp = 0xbe8967fc pc = 0x408e7c88
[task 2019-04-23T19:23:34.019Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.019Z] 19:23:33 INFO - 79 dalvik-heap (deleted) + 0x2a6
[task 2019-04-23T19:23:34.020Z] 19:23:33 INFO - sp = 0xbe896804 pc = 0x414b52a8
[task 2019-04-23T19:23:34.020Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.020Z] 19:23:33 INFO - 80 dalvik-heap (deleted) + 0x2b7512
[task 2019-04-23T19:23:34.020Z] 19:23:33 INFO - sp = 0xbe896808 pc = 0x4176c514
[task 2019-04-23T19:23:34.021Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.021Z] 19:23:33 INFO - 81 libdvm.so + 0x5ff7d
[task 2019-04-23T19:23:34.022Z] 19:23:33 INFO - sp = 0xbe896810 pc = 0x4089df7f
[task 2019-04-23T19:23:34.023Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.024Z] 19:23:33 INFO - 82 dalvik-heap (deleted) + 0x2ac006
[task 2019-04-23T19:23:34.024Z] 19:23:33 INFO - sp = 0xbe896818 pc = 0x41761008
[task 2019-04-23T19:23:34.024Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.024Z] 19:23:33 INFO - 83 dalvik-heap (deleted) + 0x2b75d6
[task 2019-04-23T19:23:34.025Z] 19:23:33 INFO - sp = 0xbe896824 pc = 0x4176c5d8
[task 2019-04-23T19:23:34.025Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.025Z] 19:23:33 INFO - 84 dalvik-heap (deleted) + 0x2ac006
[task 2019-04-23T19:23:34.025Z] 19:23:33 INFO - sp = 0xbe896828 pc = 0x41761008
[task 2019-04-23T19:23:34.026Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.026Z] 19:23:33 INFO - 85 libdvm.so + 0xa9c86
[task 2019-04-23T19:23:34.026Z] 19:23:33 INFO - sp = 0xbe89682c pc = 0x408e7c88
[task 2019-04-23T19:23:34.026Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.026Z] 19:23:33 INFO - 86 libdvm.so + 0x713a7
[task 2019-04-23T19:23:34.027Z] 19:23:33 INFO - sp = 0xbe896830 pc = 0x408af3a9
[task 2019-04-23T19:23:34.027Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.027Z] 19:23:33 INFO - 87 dalvik-heap (deleted) + 0x2b75d6
[task 2019-04-23T19:23:34.027Z] 19:23:33 INFO - sp = 0xbe89683c pc = 0x4176c5d8
[task 2019-04-23T19:23:34.027Z] 19:23:33 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.027Z] 19:23:33 INFO - 88 libdvm.so + 0x7227d
[task 2019-04-23T19:23:34.027Z] 19:23:34 INFO - sp = 0xbe896840 pc = 0x408b027f
[task 2019-04-23T19:23:34.028Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.028Z] 19:23:34 INFO - 89 libdvm.so + 0xa9c86
[task 2019-04-23T19:23:34.028Z] 19:23:34 INFO - sp = 0xbe896844 pc = 0x408e7c88
[task 2019-04-23T19:23:34.028Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.028Z] 19:23:34 INFO - 90 libdvm.so + 0xa9c86
[task 2019-04-23T19:23:34.029Z] 19:23:34 INFO - sp = 0xbe896848 pc = 0x408e7c88
[task 2019-04-23T19:23:34.029Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.029Z] 19:23:34 INFO - 91 dalvik-heap (deleted) + 0x2b75d6
[task 2019-04-23T19:23:34.029Z] 19:23:34 INFO - sp = 0xbe89684c pc = 0x4176c5d8
[task 2019-04-23T19:23:34.029Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.029Z] 19:23:34 INFO - 92 libdvm.so + 0xae19e
[task 2019-04-23T19:23:34.029Z] 19:23:34 INFO - sp = 0xbe896850 pc = 0x408ec1a0
[task 2019-04-23T19:23:34.030Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.030Z] 19:23:34 INFO - 93 libdvm.so + 0x29dee
[task 2019-04-23T19:23:34.030Z] 19:23:34 INFO - sp = 0xbe896858 pc = 0x40867df0
[task 2019-04-23T19:23:34.030Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.030Z] 19:23:34 INFO - 94 dalvik-heap (deleted) + 0x5c626
[task 2019-04-23T19:23:34.030Z] 19:23:34 INFO - sp = 0xbe896868 pc = 0x41511628
[task 2019-04-23T19:23:34.030Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.031Z] 19:23:34 INFO - 95 dalvik-LinearAlloc (deleted) + 0xa79b6
[task 2019-04-23T19:23:34.031Z] 19:23:34 INFO - sp = 0xbe89686c pc = 0x4c50e9b8
[task 2019-04-23T19:23:34.031Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.031Z] 19:23:34 INFO - 96 dalvik-heap (deleted) + 0x2b75d6
[task 2019-04-23T19:23:34.031Z] 19:23:34 INFO - sp = 0xbe896874 pc = 0x4176c5d8
[task 2019-04-23T19:23:34.033Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.034Z] 19:23:34 INFO - 97 dalvik-heap (deleted) + 0x2b74fe
[task 2019-04-23T19:23:34.036Z] 19:23:34 INFO - sp = 0xbe896878 pc = 0x4176c500
[task 2019-04-23T19:23:34.037Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.039Z] 19:23:34 INFO - 98 dalvik-heap (deleted) + 0x2a6
[task 2019-04-23T19:23:34.040Z] 19:23:34 INFO - sp = 0xbe89687c pc = 0x414b52a8
[task 2019-04-23T19:23:34.041Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.043Z] 19:23:34 INFO - 99 libdvm.so + 0x67aa1
[task 2019-04-23T19:23:34.045Z] 19:23:34 INFO - sp = 0xbe896888 pc = 0x408a5aa3
[task 2019-04-23T19:23:34.046Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.047Z] 19:23:34 INFO - 100 dalvik-heap (deleted) + 0x2a6
[task 2019-04-23T19:23:34.049Z] 19:23:34 INFO - sp = 0xbe89688c pc = 0x414b52a8
[task 2019-04-23T19:23:34.049Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.053Z] 19:23:34 INFO - 101 dalvik-LinearAlloc (deleted) + 0xbe3e
[task 2019-04-23T19:23:34.054Z] 19:23:34 INFO - sp = 0xbe896894 pc = 0x4c472e40
[task 2019-04-23T19:23:34.056Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.058Z] 19:23:34 INFO - 102 core.odex + 0xb3418
[task 2019-04-23T19:23:34.059Z] 19:23:34 INFO - sp = 0xbe896898 pc = 0x4d51b41a
[task 2019-04-23T19:23:34.061Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.062Z] 19:23:34 INFO - 103 dalvik-heap (deleted) + 0x4306
[task 2019-04-23T19:23:34.062Z] 19:23:34 INFO - sp = 0xbe8968a4 pc = 0x414b9308
[task 2019-04-23T19:23:34.065Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.065Z] 19:23:34 INFO - 104 libdvm.so + 0x1ddbe
[task 2019-04-23T19:23:34.067Z] 19:23:34 INFO - sp = 0xbe8968a8 pc = 0x4085bdc0
[task 2019-04-23T19:23:34.067Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.068Z] 19:23:34 INFO - 105 libdvm.so + 0x27062
[task 2019-04-23T19:23:34.070Z] 19:23:34 INFO - sp = 0xbe8968b8 pc = 0x40865064
[task 2019-04-23T19:23:34.072Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.075Z] 19:23:34 INFO - 106 libdvm.so + 0x2de9a
[task 2019-04-23T19:23:34.076Z] 19:23:34 INFO - sp = 0xbe8968c4 pc = 0x4086be9c
[task 2019-04-23T19:23:34.077Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.079Z] 19:23:34 INFO - 107 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.079Z] 19:23:34 INFO - sp = 0xbe8968d8 pc = 0x4c503c30
[task 2019-04-23T19:23:34.081Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.083Z] 19:23:34 INFO - 108 libdvm.so + 0x2b582
[task 2019-04-23T19:23:34.083Z] 19:23:34 INFO - sp = 0xbe8968e0 pc = 0x40869584
[task 2019-04-23T19:23:34.084Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.088Z] 19:23:34 INFO - 109 libdvm.so + 0x8e7f3
[task 2019-04-23T19:23:34.088Z] 19:23:34 INFO - sp = 0xbe8968e4 pc = 0x408cc7f5
[task 2019-04-23T19:23:34.093Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.093Z] 19:23:34 INFO - 110 libdvm.so + 0x8e668
[task 2019-04-23T19:23:34.093Z] 19:23:34 INFO - sp = 0xbe8968e8 pc = 0x408cc66a
[task 2019-04-23T19:23:34.093Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.093Z] 19:23:34 INFO - 111 framework.odex + 0x66b635
[task 2019-04-23T19:23:34.094Z] 19:23:34 INFO - sp = 0xbe896958 pc = 0x4e13d637
[task 2019-04-23T19:23:34.096Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.097Z] 19:23:34 INFO - 112 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.099Z] 19:23:34 INFO - sp = 0xbe896964 pc = 0x4c503c30
[task 2019-04-23T19:23:34.102Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.102Z] 19:23:34 INFO - 113 libdvm.so + 0x5fcbf
[task 2019-04-23T19:23:34.104Z] 19:23:34 INFO - sp = 0xbe896970 pc = 0x4089dcc1
[task 2019-04-23T19:23:34.105Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.105Z] 19:23:34 INFO - 114 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.109Z] 19:23:34 INFO - sp = 0xbe896978 pc = 0x4c503c30
[task 2019-04-23T19:23:34.109Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.109Z] 19:23:34 INFO - 115 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.112Z] 19:23:34 INFO - sp = 0xbe896980 pc = 0x4c503c30
[task 2019-04-23T19:23:34.112Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.113Z] 19:23:34 INFO - 116 libdvm.so + 0x8e6bb
[task 2019-04-23T19:23:34.114Z] 19:23:34 INFO - sp = 0xbe896994 pc = 0x408cc6bd
[task 2019-04-23T19:23:34.114Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.117Z] 19:23:34 INFO - 117 libdvm.so + 0x499ad
[task 2019-04-23T19:23:34.118Z] 19:23:34 INFO - sp = 0xbe8969a0 pc = 0x408879af
[task 2019-04-23T19:23:34.119Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.119Z] 19:23:34 INFO - 118 libdvm.so + 0x8e6bb
[task 2019-04-23T19:23:34.123Z] 19:23:34 INFO - sp = 0xbe8969b4 pc = 0x408cc6bd
[task 2019-04-23T19:23:34.123Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.124Z] 19:23:34 INFO - 119 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.124Z] 19:23:34 INFO - sp = 0xbe8969b8 pc = 0x4c503c30
[task 2019-04-23T19:23:34.124Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.125Z] 19:23:34 INFO - 120 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.125Z] 19:23:34 INFO - sp = 0xbe8969c0 pc = 0x4c503c30
[task 2019-04-23T19:23:34.126Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.130Z] 19:23:34 INFO - 121 libdvm.so + 0x3cb3d
[task 2019-04-23T19:23:34.131Z] 19:23:34 INFO - sp = 0xbe8969c8 pc = 0x4087ab3f
[task 2019-04-23T19:23:34.131Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.131Z] 19:23:34 INFO - 122 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.131Z] 19:23:34 INFO - sp = 0xbe8969d0 pc = 0x4c503c30
[task 2019-04-23T19:23:34.131Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.134Z] 19:23:34 INFO - 123 libdvm.so + 0x8e7f7
[task 2019-04-23T19:23:34.134Z] 19:23:34 INFO - sp = 0xbe8969d4 pc = 0x408cc7f9
[task 2019-04-23T19:23:34.134Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.134Z] 19:23:34 INFO - 124 libdvm.so + 0xa43b7
[task 2019-04-23T19:23:34.137Z] 19:23:34 INFO - sp = 0xbe8969dc pc = 0x408e23b9
[task 2019-04-23T19:23:34.137Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.138Z] 19:23:34 INFO - 125 libandroid_runtime.so + 0x8d6ff
[task 2019-04-23T19:23:34.140Z] 19:23:34 INFO - sp = 0xbe8969e4 pc = 0x40193701
[task 2019-04-23T19:23:34.141Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.143Z] 19:23:34 INFO - 126 app_process + 0x13a2
[task 2019-04-23T19:23:34.145Z] 19:23:34 INFO - sp = 0xbe8969e8 pc = 0x2a0013a4
[task 2019-04-23T19:23:34.147Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.148Z] 19:23:34 INFO - 127 libdvm.so + 0x3cad3
[task 2019-04-23T19:23:34.150Z] 19:23:34 INFO - sp = 0xbe8969ec pc = 0x4087aad5
[task 2019-04-23T19:23:34.152Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.153Z] 19:23:34 INFO - 128 libandroid_runtime.so + 0x8dcc5
[task 2019-04-23T19:23:34.155Z] 19:23:34 INFO - sp = 0xbe8969fc pc = 0x40193cc7
[task 2019-04-23T19:23:34.158Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.159Z] 19:23:34 INFO - 129 libandroid_runtime.so + 0x4b68f
[task 2019-04-23T19:23:34.161Z] 19:23:34 INFO - sp = 0xbe896a08 pc = 0x40151691
[task 2019-04-23T19:23:34.162Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.164Z] 19:23:34 INFO - 130 app_process + 0x13a2
[task 2019-04-23T19:23:34.167Z] 19:23:34 INFO - sp = 0xbe896a14 pc = 0x2a0013a4
[task 2019-04-23T19:23:34.169Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.173Z] 19:23:34 INFO - 131 libandroid_runtime.so + 0x4c311
[task 2019-04-23T19:23:34.173Z] 19:23:34 INFO - sp = 0xbe896a18 pc = 0x40152313
[task 2019-04-23T19:23:34.173Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.174Z] 19:23:34 INFO - 132 dalvik-LinearAlloc (deleted) + 0x9cc2e
[task 2019-04-23T19:23:34.175Z] 19:23:34 INFO - sp = 0xbe896a1c pc = 0x4c503c30
[task 2019-04-23T19:23:34.175Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.177Z] 19:23:34 INFO - 133 libc.so + 0x105f5
[task 2019-04-23T19:23:34.178Z] 19:23:34 INFO - sp = 0xbe896a48 pc = 0x400375f7
[task 2019-04-23T19:23:34.179Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.180Z] 19:23:34 INFO - 134 linker + 0x1064e
[task 2019-04-23T19:23:34.182Z] 19:23:34 INFO - sp = 0xbe896a4c pc = 0x40010650
[task 2019-04-23T19:23:34.182Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.183Z] 19:23:34 INFO - 135 libutils.so + 0x128f7
[task 2019-04-23T19:23:34.184Z] 19:23:34 INFO - sp = 0xbe896a60 pc = 0x400af8f9
[task 2019-04-23T19:23:34.186Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.187Z] 19:23:34 INFO - 136 libutils.so + 0x1255d
[task 2019-04-23T19:23:34.190Z] 19:23:34 INFO - sp = 0xbe896a68 pc = 0x400af55f
[task 2019-04-23T19:23:34.191Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.193Z] 19:23:34 INFO - 137 libutils.so + 0x12951
[task 2019-04-23T19:23:34.195Z] 19:23:34 INFO - sp = 0xbe896a78 pc = 0x400af953
[task 2019-04-23T19:23:34.197Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.200Z] 19:23:34 INFO - 138 libandroid_runtime.so + 0x4b969
[task 2019-04-23T19:23:34.202Z] 19:23:34 INFO - sp = 0xbe896a90 pc = 0x4015196b
[task 2019-04-23T19:23:34.203Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.205Z] 19:23:34 INFO - 139 app_process + 0x12f6
[task 2019-04-23T19:23:34.206Z] 19:23:34 INFO - sp = 0xbe896a98 pc = 0x2a0012f8
[task 2019-04-23T19:23:34.207Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.209Z] 19:23:34 INFO - 140 app_process + 0x2f6e
[task 2019-04-23T19:23:34.210Z] 19:23:34 INFO - sp = 0xbe896a9c pc = 0x2a002f70
[task 2019-04-23T19:23:34.212Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.214Z] 19:23:34 INFO - 141 app_process + 0x105d
[task 2019-04-23T19:23:34.217Z] 19:23:34 INFO - sp = 0xbe896ab0 pc = 0x2a00105f
[task 2019-04-23T19:23:34.218Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.220Z] 19:23:34 INFO - 142 app_process + 0x1368
[task 2019-04-23T19:23:34.221Z] 19:23:34 INFO - sp = 0xbe896ac8 pc = 0x2a00136a
[task 2019-04-23T19:23:34.222Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.224Z] 19:23:34 INFO - 143 app_process + 0x12f6
[task 2019-04-23T19:23:34.225Z] 19:23:34 INFO - sp = 0xbe896acc pc = 0x2a0012f8
[task 2019-04-23T19:23:34.227Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.228Z] 19:23:34 INFO - 144 app_process + 0x1371
[task 2019-04-23T19:23:34.230Z] 19:23:34 INFO - sp = 0xbe896ad0 pc = 0x2a001373
[task 2019-04-23T19:23:34.231Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.232Z] 19:23:34 INFO - 145 app_process + 0x1387
[task 2019-04-23T19:23:34.234Z] 19:23:34 INFO - sp = 0xbe896ad4 pc = 0x2a001389
[task 2019-04-23T19:23:34.235Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.237Z] 19:23:34 INFO - 146 app_process + 0x1395
[task 2019-04-23T19:23:34.238Z] 19:23:34 INFO - sp = 0xbe896ad8 pc = 0x2a001397
[task 2019-04-23T19:23:34.239Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.241Z] 19:23:34 INFO - 147 app_process + 0x2e06
[task 2019-04-23T19:23:34.242Z] 19:23:34 INFO - sp = 0xbe896adc pc = 0x2a002e08
[task 2019-04-23T19:23:34.243Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.245Z] 19:23:34 INFO - 148 libandroid_runtime.so + 0xa2446
[task 2019-04-23T19:23:34.247Z] 19:23:34 INFO - sp = 0xbe896ae0 pc = 0x401a8448
[task 2019-04-23T19:23:34.250Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.252Z] 19:23:34 INFO - 149 linker + 0xfed6
[task 2019-04-23T19:23:34.255Z] 19:23:34 INFO - sp = 0xbe896b10 pc = 0x4000fed8
[task 2019-04-23T19:23:34.257Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.258Z] 19:23:34 INFO - 150 linker + 0x236
[task 2019-04-23T19:23:34.259Z] 19:23:34 INFO - sp = 0xbe896b24 pc = 0x40000238
[task 2019-04-23T19:23:34.262Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.263Z] 19:23:34 INFO - 151 linker + 0x146
[task 2019-04-23T19:23:34.266Z] 19:23:34 INFO - sp = 0xbe896b28 pc = 0x40000148
[task 2019-04-23T19:23:34.268Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.269Z] 19:23:34 INFO - 152 linker + 0x2ee
[task 2019-04-23T19:23:34.270Z] 19:23:34 INFO - sp = 0xbe896b34 pc = 0x400002f0
[task 2019-04-23T19:23:34.271Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.272Z] 19:23:34 INFO - 153 linker + 0x2fa
[task 2019-04-23T19:23:34.273Z] 19:23:34 INFO - sp = 0xbe896b38 pc = 0x400002fc
[task 2019-04-23T19:23:34.274Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.276Z] 19:23:34 INFO - 154 linker + 0xffe6
[task 2019-04-23T19:23:34.277Z] 19:23:34 INFO - sp = 0xbe896b3c pc = 0x4000ffe8
[task 2019-04-23T19:23:34.279Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.280Z] 19:23:34 INFO - 155 linker + 0x9de
[task 2019-04-23T19:23:34.281Z] 19:23:34 INFO - sp = 0xbe896b40 pc = 0x400009e0
[task 2019-04-23T19:23:34.283Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.284Z] 19:23:34 INFO - 156 linker + 0x336
[task 2019-04-23T19:23:34.285Z] 19:23:34 INFO - sp = 0xbe896b48 pc = 0x40000338
[task 2019-04-23T19:23:34.286Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.288Z] 19:23:34 INFO - 157 linker + 0xfece
[task 2019-04-23T19:23:34.289Z] 19:23:34 INFO - sp = 0xbe896b58 pc = 0x4000fed0
[task 2019-04-23T19:23:34.290Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.292Z] 19:23:34 INFO - 158 libc.so + 0x119fb
[task 2019-04-23T19:23:34.293Z] 19:23:34 INFO - sp = 0xbe896b64 pc = 0x400389fd
[task 2019-04-23T19:23:34.294Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.296Z] 19:23:34 INFO - 159 libc.so + 0x47fd6
[task 2019-04-23T19:23:34.297Z] 19:23:34 INFO - sp = 0xbe896b7c pc = 0x4006efd8
[task 2019-04-23T19:23:34.298Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.299Z] 19:23:34 INFO - 160 app_process + 0xe87
[task 2019-04-23T19:23:34.300Z] 19:23:34 INFO - sp = 0xbe896b80 pc = 0x2a000e89
[task 2019-04-23T19:23:34.302Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.303Z] 19:23:34 INFO - 161 libc.so + 0xdb4f
[task 2019-04-23T19:23:34.305Z] 19:23:34 INFO - sp = 0xbe896b90 pc = 0x40034b51
[task 2019-04-23T19:23:34.306Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.307Z] 19:23:34 INFO - 162 app_process + 0xd7e
[task 2019-04-23T19:23:34.309Z] 19:23:34 INFO - sp = 0xbe896ba8 pc = 0x2a000d80
[task 2019-04-23T19:23:34.311Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.312Z] 19:23:34 INFO - 163 app_process + 0x2de6
[task 2019-04-23T19:23:34.313Z] 19:23:34 INFO - sp = 0xbe896bac pc = 0x2a002de8
[task 2019-04-23T19:23:34.315Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.317Z] 19:23:34 INFO - 164 app_process + 0x2dee
[task 2019-04-23T19:23:34.318Z] 19:23:34 INFO - sp = 0xbe896bb0 pc = 0x2a002df0
[task 2019-04-23T19:23:34.320Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.321Z] 19:23:34 INFO - 165 app_process + 0x2df6
[task 2019-04-23T19:23:34.323Z] 19:23:34 INFO - sp = 0xbe896bb4 pc = 0x2a002df8
[task 2019-04-23T19:23:34.324Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.326Z] 19:23:34 INFO - 166 linker + 0x1847
[task 2019-04-23T19:23:34.327Z] 19:23:34 INFO - sp = 0xbe896bc0 pc = 0x40001849
[task 2019-04-23T19:23:34.329Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.331Z] 19:23:34 INFO - 167 app_process + 0x32
[task 2019-04-23T19:23:34.333Z] 19:23:34 INFO - sp = 0xbe896c34 pc = 0x2a000034
[task 2019-04-23T19:23:34.336Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.342Z] 19:23:34 INFO - 168 0x3ffffffe
[task 2019-04-23T19:23:34.343Z] 19:23:34 INFO - sp = 0xbe896c4c pc = 0x40000000
[task 2019-04-23T19:23:34.344Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.346Z] 19:23:34 INFO - 169 app_process + 0xd1a
[task 2019-04-23T19:23:34.348Z] 19:23:34 INFO - sp = 0xbe896c5c pc = 0x2a000d1c
[task 2019-04-23T19:23:34.350Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.351Z] 19:23:34 INFO - 170 org.mozilla.fennec_aurora-1.apk + 0x2be6770
[task 2019-04-23T19:23:34.353Z] 19:23:34 INFO - sp = 0xbe896cc4 pc = 0x6d2e6772
[task 2019-04-23T19:23:34.354Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.356Z] 19:23:34 INFO - 171 org.mozilla.fennec_aurora-1.apk + 0x1f97a6d
[task 2019-04-23T19:23:34.358Z] 19:23:34 INFO - sp = 0xbe896cc8 pc = 0x6c697a6f
[task 2019-04-23T19:23:34.359Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.361Z] 19:23:34 INFO - 172 dalvik-LinearAlloc (deleted) + 0x18d44a
[task 2019-04-23T19:23:34.362Z] 19:23:34 INFO - sp = 0xbe896d4c pc = 0x4c5f444c
[task 2019-04-23T19:23:34.363Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.365Z] 19:23:34 INFO - 173 dalvik-heap (deleted) + 0x6f247
[task 2019-04-23T19:23:34.366Z] 19:23:34 INFO - sp = 0xbe896d50 pc = 0x41524249
[task 2019-04-23T19:23:34.368Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.369Z] 19:23:34 INFO - 174 icudt50l.dat + 0x5a0950
[task 2019-04-23T19:23:34.370Z] 19:23:34 INFO - sp = 0xbe896d54 pc = 0x505f5952
[task 2019-04-23T19:23:34.372Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.373Z] 19:23:34 INFO - 175 data@[e-mail] + 0x540e3f
[task 2019-04-23T19:23:34.374Z] 19:23:34 INFO - sp = 0xbe896d74 pc = 0x52444e41
[task 2019-04-23T19:23:34.376Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.378Z] 19:23:34 INFO - 176 libxul.so!mozilla::dom::StructuredCloneHolder::ReadFromBuffer(nsISupports*, JSContext*, JSStructuredCloneData&, unsigned int, JS::MutableHandle<JS::Value>, mozilla::ErrorResult&) [StructuredCloneHolder.cpp:99ad1e5017af4029d97661a835ab95f78a5cb2b8 : 0 + 0xa]
[task 2019-04-23T19:23:34.380Z] 19:23:34 INFO - sp = 0xbe896d7c pc = 0x544f4f42
[task 2019-04-23T19:23:34.386Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.386Z] 19:23:34 INFO - 177 icudt50l.dat + 0x7c5f4a
[task 2019-04-23T19:23:34.386Z] 19:23:34 INFO - sp = 0xbe896d80 pc = 0x4f474f4c
[task 2019-04-23T19:23:34.386Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.386Z] 19:23:34 INFO - 178 libLLVM.so + 0x32113b
[task 2019-04-23T19:23:34.387Z] 19:23:34 INFO - sp = 0xbe896d84 pc = 0x4100313d
[task 2019-04-23T19:23:34.388Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.389Z] 19:23:34 INFO - 179 icudt50l.dat + 0x87544c
[task 2019-04-23T19:23:34.390Z] 19:23:34 INFO - sp = 0xbe896d88 pc = 0x4f52444e
[task 2019-04-23T19:23:34.391Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.392Z] 19:23:34 INFO - 180 dalvik-jit-code-cache (deleted) + 0x74447
[task 2019-04-23T19:23:34.393Z] 19:23:34 INFO - sp = 0xbe896d8c pc = 0x525f4449
[task 2019-04-23T19:23:34.395Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.397Z] 19:23:34 INFO - 181 data@[e-mail] + 0x540e3f
[task 2019-04-23T19:23:34.399Z] 19:23:34 INFO - sp = 0xbe896d9c pc = 0x52444e41
[task 2019-04-23T19:23:34.400Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.403Z] 19:23:34 INFO - 182 dalvik-heap (deleted) + 0x408033f
[task 2019-04-23T19:23:34.404Z] 19:23:34 INFO - sp = 0xbe896da4 pc = 0x45535341
[task 2019-04-23T19:23:34.406Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.407Z] 19:23:34 INFO - 183 libLLVM.so + 0x32506e
[task 2019-04-23T19:23:34.409Z] 19:23:34 INFO - sp = 0xbe896db4 pc = 0x41007070
[task 2019-04-23T19:23:34.411Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.413Z] 19:23:34 INFO - 184 icudt50l.dat + 0x87544c
[task 2019-04-23T19:23:34.414Z] 19:23:34 INFO - sp = 0xbe896db8 pc = 0x4f52444e
[task 2019-04-23T19:23:34.415Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.417Z] 19:23:34 INFO - 185 dalvik-heap (deleted) + 0x313f447
[task 2019-04-23T19:23:34.419Z] 19:23:34 INFO - sp = 0xbe896dbc pc = 0x445f4449
[task 2019-04-23T19:23:34.421Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.423Z] 19:23:34 INFO - 186 framework.odex + 0x93e05f
[task 2019-04-23T19:23:34.425Z] 19:23:34 INFO - sp = 0xbe896dc8 pc = 0x4e410061
[task 2019-04-23T19:23:34.426Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.427Z] 19:23:34 INFO - 187 dalvik-bitmap-1 (deleted) + 0x40242
[task 2019-04-23T19:23:34.429Z] 19:23:34 INFO - sp = 0xbe896dcc pc = 0x494f5244
[task 2019-04-23T19:23:34.431Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.433Z] 19:23:34 INFO - 188 libxul.so!nsFrameLoader::LazyWidth() const [nsFrameLoader.cpp:99ad1e5017af4029d97661a835ab95f78a5cb2b8 : 2554 + 0x2]
[task 2019-04-23T19:23:34.435Z] 19:23:34 INFO - sp = 0xbe896dd0 pc = 0x54535f44
[task 2019-04-23T19:23:34.437Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.439Z] 19:23:34 INFO - 189 dalvik-heap (deleted) + 0x5f6024d
[task 2019-04-23T19:23:34.441Z] 19:23:34 INFO - sp = 0xbe896dd4 pc = 0x4741524f
[task 2019-04-23T19:23:34.443Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.445Z] 19:23:34 INFO - 190 libLLVM.so + 0x324565
[task 2019-04-23T19:23:34.446Z] 19:23:34 INFO - sp = 0xbe896de0 pc = 0x41006567
[task 2019-04-23T19:23:34.448Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.450Z] 19:23:34 INFO - 191 dalvik-aux-structure (deleted) + 0x74f4b
[task 2019-04-23T19:23:34.451Z] 19:23:34 INFO - sp = 0xbe896de8 pc = 0x4e554f4d
[task 2019-04-23T19:23:34.453Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.454Z] 19:23:34 INFO - 192 dalvik-bitmap-1 (deleted) + 0x40052
[task 2019-04-23T19:23:34.456Z] 19:23:34 INFO - sp = 0xbe896dec pc = 0x494f5054
[task 2019-04-23T19:23:34.458Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.460Z] 19:23:34 INFO - 193 icudt50l.dat + 0x845bfe
[task 2019-04-23T19:23:34.465Z] 19:23:34 INFO - sp = 0xbe896dfc pc = 0x4f4f4c00
[task 2019-04-23T19:23:34.466Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.467Z] 19:23:34 INFO - 194 icudt50l.dat + 0x826f4e
[task 2019-04-23T19:23:34.469Z] 19:23:34 INFO - sp = 0xbe896e00 pc = 0x4f4d5f50
[task 2019-04-23T19:23:34.470Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.471Z] 19:23:34 INFO - 195 icudt50l.dat + 0x4efe53
[task 2019-04-23T19:23:34.473Z] 19:23:34 INFO - sp = 0xbe896e04 pc = 0x50544e55
[task 2019-04-23T19:23:34.474Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.479Z] 19:23:34 INFO - 196 libxul.so!bool mozilla::BinarySearchIf<nsID const*, unsigned int nsTArray_Impl<nsID, nsTArrayInfallibleAllocator>::BinaryIndexOf<nsID, int (nsID const&, nsID const&)>(nsID const&, int ( const&)(nsID const&, nsID const&)) const::{lambda(nsID const&)#1}>(nsID const* const&, unsigned int, unsigned int, unsigned int nsTArray_Impl<nsID, nsTArrayInfallibleAllocator>::BinaryIndexOf<nsID, int (nsID const&, nsID const&)>(nsID const&, int ( const&)(nsID const&, nsID const&)) const::{lambda(nsID const&)#1} const&, unsigned int*) [BinarySearch.h:99ad1e5017af4029d97661a835ab95f78a5cb2b8 : 0 + 0xd]
[task 2019-04-23T19:23:34.481Z] 19:23:34 INFO - sp = 0xbe896e08 pc = 0x544e494f
[task 2019-04-23T19:23:34.482Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.483Z] 19:23:34 INFO - 197 icudt50l.dat + 0x771060
[task 2019-04-23T19:23:34.484Z] 19:23:34 INFO - sp = 0xbe896e14 pc = 0x4f420062
[task 2019-04-23T19:23:34.485Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.486Z] 19:23:34 INFO - 198 dalvik-card-table (deleted) + 0xd544d
[task 2019-04-23T19:23:34.488Z] 19:23:34 INFO - sp = 0xbe896e18 pc = 0x4c43544f
[task 2019-04-23T19:23:34.489Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.491Z] 19:23:34 INFO - 199 icudt50l.dat + 0x4e033f
[task 2019-04-23T19:23:34.492Z] 19:23:34 INFO - sp = 0xbe896e1c pc = 0x50535341
[task 2019-04-23T19:23:34.494Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.495Z] 19:23:34 INFO - 200 org.mozilla.fennec_aurora-1.apk + 0x2f17264
[task 2019-04-23T19:23:34.497Z] 19:23:34 INFO - sp = 0xbe896e2c pc = 0x6d617266
[task 2019-04-23T19:23:34.498Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.500Z] 19:23:34 INFO - 201 org.mozilla.fennec_aurora-1.apk + 0x1026f75
[task 2019-04-23T19:23:34.501Z] 19:23:34 INFO - sp = 0xbe896e4c pc = 0x6b726f77
[task 2019-04-23T19:23:34.503Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.504Z] 19:23:34 INFO - 202 org.mozilla.fennec_aurora-1.apk + 0x2f17264
[task 2019-04-23T19:23:34.506Z] 19:23:34 INFO - sp = 0xbe896e68 pc = 0x6d617266
[task 2019-04-23T19:23:34.520Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.523Z] 19:23:34 INFO - 203 org.mozilla.fennec_aurora-1.apk + 0x1026f75
[task 2019-04-23T19:23:34.527Z] 19:23:34 INFO - sp = 0xbe896e90 pc = 0x6b726f77
[task 2019-04-23T19:23:34.532Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.532Z] 19:23:34 INFO - 204 org.mozilla.fennec_aurora-1.apk + 0x2f57471
[task 2019-04-23T19:23:34.534Z] 19:23:34 INFO - sp = 0xbe896ea0 pc = 0x6d657473
[task 2019-04-23T19:23:34.535Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.537Z] 19:23:34 INFO - 205 org.mozilla.fennec_aurora-1.apk + 0x1026f75
[task 2019-04-23T19:23:34.539Z] 19:23:34 INFO - sp = 0xbe896eb4 pc = 0x6b726f77
[task 2019-04-23T19:23:34.540Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.542Z] 19:23:34 INFO - 206 org.mozilla.fennec_aurora-1.apk + 0x2f57471
[task 2019-04-23T19:23:34.543Z] 19:23:34 INFO - sp = 0xbe896ec0 pc = 0x6d657473
[task 2019-04-23T19:23:34.545Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.546Z] 19:23:34 INFO - 207 org.mozilla.fennec_aurora-1.apk + 0x2ff632b
[task 2019-04-23T19:23:34.547Z] 19:23:34 INFO - sp = 0xbe896ed8 pc = 0x6d6f632d
[task 2019-04-23T19:23:34.548Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.549Z] 19:23:34 INFO - 208 org.mozilla.fennec_aurora-1.apk + 0x2f17264
[task 2019-04-23T19:23:34.551Z] 19:23:34 INFO - sp = 0xbe896eec pc = 0x6d617266
[task 2019-04-23T19:23:34.552Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.554Z] 19:23:34 INFO - 209 org.mozilla.fennec_aurora-1.apk + 0x2fd6f61
[task 2019-04-23T19:23:34.555Z] 19:23:34 INFO - sp = 0xbe896f1c pc = 0x6d6d6f63
[task 2019-04-23T19:23:34.557Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.558Z] 19:23:34 INFO - 210 org.mozilla.fennec_aurora-1.apk + 0x1026f75
[task 2019-04-23T19:23:34.560Z] 19:23:34 INFO - sp = 0xbe896f34 pc = 0x6b726f77
[task 2019-04-23T19:23:34.561Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.563Z] 19:23:34 INFO - 211 org.mozilla.fennec_aurora-1.apk + 0x1ff702c
[task 2019-04-23T19:23:34.565Z] 19:23:34 INFO - sp = 0xbe896f40 pc = 0x6c6f702e
[task 2019-04-23T19:23:34.565Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.565Z] 19:23:34 INFO - 212 org.mozilla.fennec_aurora-1.apk + 0x2f17264
[task 2019-04-23T19:23:34.566Z] 19:23:34 INFO - sp = 0xbe896f54 pc = 0x6d617266
[task 2019-04-23T19:23:34.568Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.570Z] 19:23:34 INFO - 213 org.mozilla.fennec_aurora-1.apk + 0x1026f75
[task 2019-04-23T19:23:34.571Z] 19:23:34 INFO - sp = 0xbe896f78 pc = 0x6b726f77
[task 2019-04-23T19:23:34.574Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.574Z] 19:23:34 INFO - 214 dalvik-heap (deleted) + 0x4090843
[task 2019-04-23T19:23:34.575Z] 19:23:34 INFO - sp = 0xbe896f8c pc = 0x45545845
[task 2019-04-23T19:23:34.576Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.577Z] 19:23:34 INFO - 215 dalvik-card-table (deleted) + 0xb4e50
[task 2019-04-23T19:23:34.578Z] 19:23:34 INFO - sp = 0xbe896f90 pc = 0x4c414e52
[task 2019-04-23T19:23:34.580Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.583Z] 19:23:34 INFO - 216 icudt50l.dat + 0x89635d
[task 2019-04-23T19:23:34.587Z] 19:23:34 INFO - sp = 0xbe896f94 pc = 0x4f54535f
[task 2019-04-23T19:23:34.590Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.592Z] 19:23:34 INFO - 217 dalvik-heap (deleted) + 0x3fbf150
[task 2019-04-23T19:23:34.594Z] 19:23:34 INFO - sp = 0xbe896f98 pc = 0x45474152
[task 2019-04-23T19:23:34.596Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.598Z] 19:23:34 INFO - 218 dalvik-heap (deleted) + 0x302f0fe
[task 2019-04-23T19:23:34.600Z] 19:23:34 INFO - sp = 0xbe896fac pc = 0x444e4100
[task 2019-04-23T19:23:34.602Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.604Z] 19:23:34 INFO - 219 dalvik-heap (deleted) + 0x2fdff50
[task 2019-04-23T19:23:34.606Z] 19:23:34 INFO - sp = 0xbe896fb0 pc = 0x44494f52
[task 2019-04-23T19:23:34.608Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.611Z] 19:23:34 INFO - 220 icudt50l.dat + 0x87605d
[task 2019-04-23T19:23:34.612Z] 19:23:34 INFO - sp = 0xbe896fb4 pc = 0x4f52505f
[task 2019-04-23T19:23:34.615Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.617Z] 19:23:34 INFO - 221 libxul.so!nsDOMTokenList::Length() [nsDOMTokenList.cpp:99ad1e5017af4029d97661a835ab95f78a5cb2b8 : 96 + 0x2]
[task 2019-04-23T19:23:34.619Z] 19:23:34 INFO - sp = 0xbe896fb8 pc = 0x54524550
[task 2019-04-23T19:23:34.621Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.622Z] 19:23:34 INFO - 222 icudt50l.dat + 0x8c6f57
[task 2019-04-23T19:23:34.624Z] 19:23:34 INFO - sp = 0xbe896fbc pc = 0x4f575f59
[task 2019-04-23T19:23:34.625Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.626Z] 19:23:34 INFO - 223 icudt50l.dat + 0x4dfb50
[task 2019-04-23T19:23:34.627Z] 19:23:34 INFO - sp = 0xbe896fc0 pc = 0x50534b52
[task 2019-04-23T19:23:34.629Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.630Z] 19:23:34 INFO - 224 data@[e-mail] + 0x540e3f
[task 2019-04-23T19:23:34.631Z] 19:23:34 INFO - sp = 0xbe896fd0 pc = 0x52444e41
[task 2019-04-23T19:23:34.632Z] 19:23:34 INFO - Found by: stack scanning
[task 2019-04-23T19:23:34.634Z] 19:23:34 INFO - 225 dalvik-mark-stack (deleted) + 0x1b7ff51
[task 2019-04-23T19:23:34.635Z] 19:23:34 INFO - sp = 0xbe896fd8 pc = 0x4b434f53

Andrew, could you please assign this to someone?

Comment 59

[Geoff Brown [:gbrown]]

Work on-going in bug 1534732.

Comment 60

[Intermittent Failures Robot]

25 failures in 499 pushes (0.05 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 24
• try: 1

Platform breakdown:

• android-em-7-0-x86_64: 24
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-04-23&endday=2019-04-23&tree=all

Comment 61

[Intermittent Failures Robot]

49 failures in 3031 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #50 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 1
• try: 4
• mozilla-central: 3
• autoland: 40
• mozilla-beta: 1

Platform breakdown:

• android-em-7-0-x86_64: 46
• android-hw-p2-8-0-arm7-api-16: 1
• android-em-4-3-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-04-22&endday=2019-04-28&tree=all

Comment 62

[Intermittent Failures Robot]

26 failures in 4131 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 19
• mozilla-inbound: 4
• mozilla-central: 2
• try: 1

Platform breakdown:

• android-em-7-0-x86_64: 25
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-04-29&endday=2019-05-05&tree=all

Comment 63

[Dorel Luca [:dluca]]

This bug has failed 32 times in the last 7 days. Occurs on android-em-7-0-x86_64 on opt build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=245716502&repo=autoland&lineNumber=3094

Comment 64

[Intermittent Failures Robot]

27 failures in 4705 pushes (0.006 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 21
• mozilla-inbound: 3
• mozilla-central: 3

Platform breakdown:

• android-em-7-0-x86_64: 26
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-05-06&endday=2019-05-12&tree=all

Comment 65

[Razvan Maries]

There are 31 total failures in the last 7 days: https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-05-12&endday=2019-05-19&tree=trunk&bug=1500977

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=247164340&repo=autoland&lineNumber=2701

Comment 66

[Intermittent Failures Robot]

37 failures in 4658 pushes (0.008 failures/push) were associated with this bug in the last 7 days.

This is the #48 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 22
• try: 4
• mozilla-inbound: 3
• mozilla-central: 6
• mozilla-beta: 2

Platform breakdown:

• android-em-7-0-x86_64: 33
• android-em-4-3-armv7-api16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-05-13&endday=2019-05-19&tree=all

Comment 67

[Intermittent Failures Robot]

47 failures in 4730 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

This is the #46 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 2
• try: 2
• mozilla-inbound: 10
• autoland: 33

Platform breakdown:

• android-em-7-0-x86_64: 45
• android-hw-p2-8-0-arm7-api-16: 1
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-05-20&endday=2019-05-26&tree=all

Comment 68

[Arthur Iakab [arthur_iakab]]

There are 56 failures in the last 7 days https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?startday=2019-05-22&endday=2019-05-29&tree=trunk&bug=1500977

Recent failure logs https://treeherder.mozilla.org/logviewer.html#?job_id=248901933&repo=autoland

Comment 69

[Geoff Brown [:gbrown]]

Almost all failures are android 7.0 mochitests: Work on-going in bug 1534732.

Comment 70

[Intermittent Failures Robot]

50 failures in 4337 pushes (0.012 failures/push) were associated with this bug in the last 7 days.

This is the #40 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 34
• mozilla-inbound: 7
• mozilla-beta: 7
• mozilla-central: 2

Platform breakdown:

• android-em-7-0-x86_64: 49
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-05-27&endday=2019-06-02&tree=all

Comment 71

[Dorel Luca [:dluca]]

This bug failed 49 times in the last 7 days. Occurs on android-em-7-0-x86_64 on opt build types.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=250702341&repo=autoland&lineNumber=4247

Comment 72

[Intermittent Failures Robot]

59 failures in 5197 pushes (0.011 failures/push) were associated with this bug in the last 7 days.

This is the #29 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 36
• mozilla-beta: 7
• mozilla-central: 5
• mozilla-inbound: 7
• try: 3
• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64: 53
• android-em-7-0-x86_64-beta: 2
• android-hw-p2-8-0-arm7-api-16: 2
• android-em-4-3-armv7-api16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-06-03&endday=2019-06-09&tree=all

Comment 73

[Intermittent Failures Robot]

15 failures in 963 pushes (0.016 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-beta: 2
• mozilla-inbound: 2
• mozilla-central: 1

Platform breakdown:

• android-em-7-0-x86_64: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-06-11&endday=2019-06-11&tree=all

Comment 74

[Andreea Pavel [:apavel]]

Geoff is this something we can disable until there is a fix in bug 1534732?

Comment 75

[Geoff Brown [:gbrown]]

Sorry, no. I think this can affect any mochitest.

Comment 76

[Andreea Pavel [:apavel]]

(In reply to Geoff Brown [:gbrown] from comment #75)

Sorry, no. I think this can affect any mochitest.

Okay, thank you.

Comment 77

[Intermittent Failures Robot]

60 failures in 4753 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #30 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-beta: 10
• autoland: 41
• mozilla-inbound: 5
• mozilla-central: 3
• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64: 56
• android-em-7-0-x86_64-beta: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-06-10&endday=2019-06-16&tree=all

Comment 78

[Intermittent Failures Robot]

38 failures in 2581 pushes (0.015 failures/push) were associated with this bug in the last 7 days.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-inbound: 5
• mozilla-central: 6
• mozilla-beta: 8
• autoland: 17
• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64: 33
• android-em-4-3-armv7-api16: 1
• android-em-7-0-x86_64-beta: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-06-17&endday=2019-06-23&tree=all

Comment 79

[Intermittent Failures Robot]

29 failures in 4409 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 14
• mozilla-central: 7
• mozilla-beta: 6
• mozilla-inbound: 1
• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64: 26
• android-em-7-0-x86_64-beta: 2
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-06-24&endday=2019-06-30&tree=all

Comment 80

[Intermittent Failures Robot]

19 failures in 509 pushes (0.037 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-release: 2
• mozilla-esr68: 3
• mozilla-inbound: 14

Platform breakdown:

• android-em-7-0-x86_64: 15
• android-em-7-0-x86_64-beta: 2
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-01&endday=2019-07-01&tree=all

Comment 81

[Geoff Brown [:gbrown]]

(In reply to Intermittent Failures Robot from comment #80)

19 failures in 509 pushes (0.037 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-release: 2
• mozilla-esr68: 3
• mozilla-inbound: 14

Platform breakdown:

• android-em-7-0-x86_64: 15
• android-em-7-0-x86_64-beta: 2
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-01&endday=2019-07-01&tree=all

Recent landings in bug 1556058 should have addressed the majority of android x86 time outs, so this is surprising.

However, some of these failures were also marked fix by commit, for the backout of bug 1547669.

Comment 82

[Intermittent Failures Robot]

27 failures in 602 pushes (0.045 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• mozilla-inbound: 23
• autoland: 2
• mozilla-release: 2

Platform breakdown:

• android-em-7-0-x86_64: 23
• android-hw-p2-8-0-arm7-api-16: 3
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-02&endday=2019-07-02&tree=all

Comment 83

[Geoff Brown [:gbrown]]

Also, many of these were related to bug 1561150 -- seems to be sorted out now.

Comment 84

[Intermittent Failures Robot]

17 failures in 175 pushes (0.097 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 17

Platform breakdown:

• android-em-7-0-x86_64: 16
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-06&endday=2019-07-06&tree=all

Comment 85

[Intermittent Failures Robot]

99 failures in 3501 pushes (0.028 failures/push) were associated with this bug in the last 7 days.

This is the #7 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-release: 5
• mozilla-esr68: 9
• mozilla-inbound: 37
• autoland: 3
• try: 45

Platform breakdown:

• android-em-7-0-x86_64: 74
• android-em-7-0-x86_64-beta: 5
• android-hw-p2-8-0-arm7-api-16: 19
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-01&endday=2019-07-07&tree=all

Comment 86

[Intermittent Failures Robot]

18 failures in 551 pushes (0.033 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 15
• mozilla-esr68: 3

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 5
• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64: 11
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-08&endday=2019-07-08&tree=all

Comment 87

[Andreea Pavel [:apavel]]

Recent occurences here are missclassifications from beta-sims and are actually bug 1563997.

Comment 88

[Intermittent Failures Robot]

20 failures in 1090 pushes (0.018 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 16
• mozilla-beta: 2
• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64: 19
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-09&endday=2019-07-09&tree=all

Comment 89

[Intermittent Failures Robot]

23 failures in 893 pushes (0.026 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-beta: 9
• try: 3
• mozilla-release: 1
• autoland: 10

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 6
• android-em-7-0-x86_64: 16
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-11&endday=2019-07-11&tree=all

Comment 90

[Intermittent Failures Robot]

76 failures in 4710 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #26 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• try: 33
• mozilla-esr68: 7
• mozilla-beta: 23
• autoland: 11
• mozilla-release: 1
• mozilla-inbound: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 18
• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64: 50
• android-em-7-0-x86_64-beta: 5
• android-hw-g5-7-0-arm7-api-16: 1
• android-em-4-3-armv7-api16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-08&endday=2019-07-14&tree=all

Comment 91

[Geoff Brown [:gbrown]]

The 07-11/07-12 spike on android-em-7-0-x86_64 was resolved by backout of bug 1562762.

Comment 92

[Geoff Brown [:gbrown]]

android-em-7-0 failures continue on beta/release/esr68 - expected until emulator is updated there.

Comment 93

[Intermittent Failures Robot]

18 failures in 4298 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 11
• mozilla-beta: 6
• mozilla-release: 1

Platform breakdown:

• android-em-7-0-x86_64-beta: 6
• android-em-7-0-x86_64: 8
• android-em-7-0-x86_64-release: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-15&endday=2019-07-21&tree=all

Comment 94

[Intermittent Failures Robot]

12 failures in 4608 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 11
• mozilla-release: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 3
• android-em-7-0-x86_64-beta: 4
• android-em-7-0-x86_64: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-22&endday=2019-07-28&tree=all

Comment 95

[Geoff Brown [:gbrown]]

This has been low frequency on trunk for 3 weeks now.

Comment 96

[Intermittent Failures Robot]

9 failures in 4439 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 4
• mozilla-central: 1
• mozilla-esr68: 4

Platform breakdown:

• android-em-7-0-x86_64-beta: 6
• android-em-4-3-armv7-api16: 1
• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-07-29&endday=2019-08-04&tree=all

Comment 97

[Intermittent Failures Robot]

3 failures in 4666 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3

Platform breakdown:

• android-em-7-0-x86_64: 2
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-08-05&endday=2019-08-11&tree=all

Comment 98

[Intermittent Failures Robot]

8 failures in 4839 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 8

Platform breakdown:

• android-em-7-0-x86_64: 5
• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64-release: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-08-12&endday=2019-08-18&tree=all

Comment 99

[Intermittent Failures Robot]

1 failures in 5073 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-08-19&endday=2019-08-25&tree=all

Comment 100

[Intermittent Failures Robot]

2 failures in 5047 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1
• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-08-26&endday=2019-09-01&tree=all

Comment 101

[Intermittent Failures Robot]

6 failures in 4408 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 6

Platform breakdown:

• android-em-7-0-x86_64-beta: 3
• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-09-02&endday=2019-09-08&tree=all

Comment 102

[Intermittent Failures Robot]

2 failures in 4637 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64: 1
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-09-09&endday=2019-09-15&tree=all

Comment 103

[Intermittent Failures Robot]

5 failures in 4350 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 5

Platform breakdown:

• android-em-7-0-x86_64-release: 2
• android-em-7-0-x86_64: 1
• android-em-4-3-armv7-api16: 1
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-09-16&endday=2019-09-22&tree=all

Comment 104

[Intermittent Failures Robot]

5 failures in 4530 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 5

Platform breakdown:

• android-em-7-0-x86_64-beta: 3
• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 105

[Intermittent Failures Robot]

2 failures in 4292 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-09-30&endday=2019-10-06&tree=all

Comment 106

[Intermittent Failures Robot]

1 failures in 3801 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-10-07&endday=2019-10-13&tree=all

Comment 107

[Intermittent Failures Robot]

2 failures in 3964 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-10-14&endday=2019-10-20&tree=all

Comment 108

[Intermittent Failures Robot]

2 failures in 4182 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-10-21&endday=2019-10-27&tree=all

Comment 109

[Intermittent Failures Robot]

2 failures in 4314 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-10-28&endday=2019-11-03&tree=all

Comment 110

[Intermittent Failures Robot]

2 failures in 4850 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-11-04&endday=2019-11-10&tree=all

Comment 111

[Intermittent Failures Robot]

21 failures in 4207 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 12
• autoland: 9

Platform breakdown:

• android-em-7-0-x86_64-beta: 6
• android-em-7-0-x86_64: 13
• android-em-7-0-x86_64-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-11-11&endday=2019-11-17&tree=all

Comment 112

[Intermittent Failures Robot]

8 failures in 5444 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 7
• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64-beta: 3
• android-em-7-0-x86_64: 4
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-11-18&endday=2019-11-24&tree=all

Comment 113

[Intermittent Failures Robot]

5 failures in 4331 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 4
• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64-beta: 2
• android-em-7-0-x86_64: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-11-25&endday=2019-12-01&tree=all

Comment 114

[Intermittent Failures Robot]

4 failures in 5144 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3
• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64: 2
• android-em-7-0-x86_64-beta: 1
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-12-02&endday=2019-12-08&tree=all

Comment 115

[Intermittent Failures Robot]

4 failures in 4867 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3
• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64-beta: 1
• android-hw-p2-8-0-arm7-api-16: 1
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-12-09&endday=2019-12-15&tree=all

Comment 116

[Intermittent Failures Robot]

2 failures in 4582 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-12-16&endday=2019-12-22&tree=all

Comment 117

[Intermittent Failures Robot]

3 failures in 1862 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3

Platform breakdown:

• android-em-7-0-x86_64-release: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2019-12-30&endday=2020-01-05&tree=all

Comment 118

[Intermittent Failures Robot]

13 failures in 4472 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 13

Platform breakdown:

• android-em-7-0-x86_64-beta: 7
• android-em-7-0-x86_64: 3
• android-em-7-0-x86_64-release: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-01-06&endday=2020-01-12&tree=all

Comment 119

[Intermittent Failures Robot]

5 failures in 3773 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 5

Platform breakdown:

• android-em-7-0-x86_64: 3
• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-01-13&endday=2020-01-19&tree=all

Comment 120

[Intermittent Failures Robot]

1 failures in 4004 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-01-20&endday=2020-01-26&tree=all

Comment 121

[Intermittent Failures Robot]

2 failures in 3364 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64: 1
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-02-03&endday=2020-02-09&tree=all

Comment 122

[Intermittent Failures Robot]

7 failures in 4690 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 4
• autoland: 3

Platform breakdown:

• android-em-7-0-x86_64-release: 2
• android-hw-p2-8-0-android-aarch64-qr: 2
• android-hw-p2-8-0-android-aarch64: 1
• android-em-7-0-x86_64: 1
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-02-10&endday=2020-02-16&tree=all

Comment 123

[Intermittent Failures Robot]

4 failures in 4045 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 4

Platform breakdown:

• android-em-7-0-x86_64: 2
• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-02-17&endday=2020-02-23&tree=all

Comment 124

[Intermittent Failures Robot]

3 failures in 5233 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3

Platform breakdown:

• android-em-7-0-x86_64-release: 1
• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-02-24&endday=2020-03-01&tree=all

Comment 125

[Intermittent Failures Robot]

3 failures in 5483 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-esr68: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 1
• android-em-7-0-x86_64: 1
• android-em-7-0-x86_64-beta: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-03-02&endday=2020-03-08&tree=all

Comment 126

[Intermittent Failures Robot]

16 failures in 913 pushes (0.018 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-central: 4
• mozilla-beta: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 16

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-03-13&endday=2020-03-13&tree=all

Comment 127

[Oana Pop-Rus]

In the last 3 days, there have been 20 failures on android-hw-p2-8-0-android-aarch64, opt and debug build type.

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=293150226&repo=autoland&lineNumber=2787

Comment 128

[Cosmin Sabou [:CosminS]]

The spike here is caused by Bug 1622155 as they run on gecko-t-bitbar-gw-unit-p2.

Comment 129

[Intermittent Failures Robot]

28 failures in 5336 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 6
• autoland: 16
• mozilla-central: 4
• mozilla-beta: 2

Platform breakdown:

• android-em-7-0-x86_64-release: 2
• android-em-7-0-x86_64-beta: 3
• android-em-7-0-x86_64: 1
• android-hw-p2-8-0-android-aarch64: 22

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-03-09&endday=2020-03-15&tree=all

Comment 130

[Geoff Brown [:gbrown]]

Thanks Cosmin.

Comment 131

[Intermittent Failures Robot]

18 failures in 908 pushes (0.02 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 18

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 9
• android-em-7-0-x86_64: 4
• android-em-7-0-x86_64-qr: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-03-18&endday=2020-03-18&tree=all

Comment 132

[Intermittent Failures Robot]

27 failures in 4955 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2
• autoland: 24
• mozilla-beta: 1

Platform breakdown:

• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64-release: 1
• android-hw-p2-8-0-android-aarch64: 16
• android-em-7-0-x86_64-qr: 5
• android-em-7-0-x86_64: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-03-16&endday=2020-03-22&tree=all

Comment 133

[Andrei Ciure[:aciure]]

There are 24 failures associated to this bug in the last 7 days. These are occurring on

• android-hw-p2-8-0-android-aarch64 - pgo and debug
• android-em-7-0-x86_64-qr - opt and debug

There are no failures in the last few days.

Comment 134

[Intermittent Failures Robot]

2 failures in 4773 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-03-23&endday=2020-03-29&tree=all

Comment 135

[Intermittent Failures Robot]

16 failures in 5015 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 6
• autoland: 10

Platform breakdown:

• android-em-7-0-x86_64-beta: 4
• android-hw-p2-8-0-android-aarch64: 10
• android-em-7-0-x86_64-release: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-03-30&endday=2020-04-05&tree=all

Comment 136

[Intermittent Failures Robot]

2 failures in 4823 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64-beta: 1
• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-04-06&endday=2020-04-12&tree=all

Comment 137

[Andreea Pavel [:apavel]]

Jean-Yves this also started spiking with bug 1607984, please take a look.

Comment 138

[Andreea Pavel [:apavel]]

We backed out due to the high frequency rate: https://hg.mozilla.org/integration/autoland/rev/cdf53efcd4c99729cd2853a9546f581e22c2cad1

Comment 139

[Intermittent Failures Robot]

3 failures in 4528 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 3

Platform breakdown:

• android-em-7-0-x86_64-qr: 1
• android-em-7-0-x86_64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-04-13&endday=2020-04-19&tree=all

Comment 140

[Intermittent Failures Robot]

5 failures in 5582 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 4
• autoland: 1

Platform breakdown:

• android-em-7-0-x86_64-beta: 2
• android-em-7-0-x86_64-release: 2
• android-em-7-0-x86_64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-04-20&endday=2020-04-26&tree=all

Comment 141

[Intermittent Failures Robot]

3 failures in 5488 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3

Platform breakdown:

• android-em-7-0-x86_64-release: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-05-11&endday=2020-05-17&tree=all

Comment 142

[Intermittent Failures Robot]

1 failures in 4727 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-05-18&endday=2020-05-24&tree=all

Comment 143

[Intermittent Failures Robot]

3 failures in 5458 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3

Platform breakdown:

• android-em-7-0-x86_64-release: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-05-25&endday=2020-05-31&tree=all

Comment 144

[Intermittent Failures Robot]

1 failures in 4949 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-06-01&endday=2020-06-07&tree=all

Comment 145

[Intermittent Failures Robot]

1 failures in 4035 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-06-15&endday=2020-06-21&tree=all

Comment 146

[Intermittent Failures Robot]

1 failures in 5252 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-06-22&endday=2020-06-28&tree=all

Comment 147

[Intermittent Failures Robot]

1 failures in 5265 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 1

Platform breakdown:

• android-em-7-0-x86_64-release: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-07-06&endday=2020-07-12&tree=all

Comment 148

[Intermittent Failures Robot]

3 failures in 4014 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 3

Platform breakdown:

• android-em-7-0-x86_64-release: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-07-13&endday=2020-07-19&tree=all

Comment 149

[Intermittent Failures Robot]

1 failures in 4289 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-07-27&endday=2020-08-02&tree=all

Comment 150

[Intermittent Failures Robot]

2 failures in 4618 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-esr68: 2

Platform breakdown:

• android-em-7-0-x86_64-release: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1500977&startday=2020-08-03&endday=2020-08-09&tree=all

Comment 151

[BugBot [:suhaib / :marco/ :calixte]]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup
For more information, please visit auto_nag documentation.