1533454 - Intermittent raptor-main TEST-UNEXPECTED-FAIL: no raptor test results were found for raptor-speedometer-fennec

Status: RESOLVED INCOMPLETE

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=232421057&repo=try

https://queue.taskcluster.net/v1/task/ZvbMt6T-T0aB286F5IhU_w/runs/0/artifacts/public/logs/live_backing.log

14:22:05 INFO - raptor-main starting fennec
14:22:05 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell am force-stop org.mozilla.firefox_beta; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:22:05 INFO - adb Granting important runtime permissions to org.mozilla.firefox_beta
14:22:06 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell pm grant org.mozilla.firefox_beta android.permission.WRITE_EXTERNAL_STORAGE; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:22:06 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell pm grant org.mozilla.firefox_beta android.permission.READ_EXTERNAL_STORAGE; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:22:06 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell pm grant org.mozilla.firefox_beta android.permission.ACCESS_COARSE_LOCATION; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:22:07 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell pm grant org.mozilla.firefox_beta android.permission.ACCESS_FINE_LOCATION; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:22:07 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell pm grant org.mozilla.firefox_beta android.permission.CAMERA; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:22:07 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell pm grant org.mozilla.firefox_beta android.permission.RECORD_AUDIO; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:22:07 INFO - adb launch_application: am start -W -n org.mozilla.firefox_beta/org.mozilla.gecko.BrowserApp -a android.intent.action.VIEW --es args "-profile /sdcard/raptor/profile --es env0 LOG_VERBOSE=1 --es env1 R_LOG_LEVEL=6" -d about:blank
14:22:08 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell am start -W -n org.mozilla.firefox_beta/org.mozilla.gecko.BrowserApp -a android.intent.action.VIEW --es args "-profile /sdcard/raptor/profile --es env0 LOG_VERBOSE=1 --es env1 R_LOG_LEVEL=6" -d about:blank; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output: Starting: Intent { act=android.intent.action.VIEW dat=about:blank cmp=org.mozilla.firefox_beta/org.mozilla.gecko.BrowserApp (has extras) }
14:22:08 INFO - Status: ok
14:22:08 INFO - Activity: org.mozilla.firefox_beta/org.mozilla.gecko.BrowserApp
14:22:08 INFO - ThisTime: 271
14:22:08 INFO - TotalTime: 271
14:22:08 INFO - WaitTime: 286
14:22:08 INFO - Complete
14:22:09 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:09] "POST / HTTP/1.1" 200 -
14:22:09 INFO - raptor-control-server received webext_status: raptor runner.js is loaded!
14:22:09 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:09] "GET /raptor-speedometer-fennec.json HTTP/1.1" 200 -
14:22:09 INFO - raptor-control-server reading test settings from raptor-speedometer-fennec.json
14:22:09 INFO - raptor-control-server sent test settings to web ext runner
14:22:09 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:09] "POST / HTTP/1.1" 200 -
14:22:09 INFO - raptor-control-server received webext_status: * pausing 30 seconds to let browser settle... *
14:22:39 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:39] "POST / HTTP/1.1" 200 -
14:22:39 INFO - raptor-control-server received webext_status: running 1 pagecycles of http://127.0.0.1:39451/Speedometer/index.html?raptor
14:22:39 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:39] "POST / HTTP/1.1" 200 -
14:22:39 INFO - raptor-control-server received webext_status: opened new empty tab 1
14:22:40 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:40] "POST / HTTP/1.1" 200 -
14:22:40 INFO - raptor-control-server received webext_status: begin pagecycle 1
14:22:41 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:41] "POST / HTTP/1.1" 200 -
14:22:41 INFO - raptor-control-server received webext_status: update tab 1
14:22:41 INFO - 127.0.0.1 - - [07/Mar/2019 14:22:41] "POST / HTTP/1.1" 200 -
14:22:41 INFO - raptor-control-server received webext_status: test tab updated 1
14:37:38 INFO - raptor-main application timed out after 933 seconds
14:37:38 INFO - adb shell_output: adb -s HT85S1A02743 wait-for-device shell am force-stop org.mozilla.firefox_beta; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
14:37:38 INFO - adb command_output: adb -s HT85S1A02743 wait-for-device pull /sdcard/raptor/profile/minidumps /tmp/tmphYYATR/minidumps, timeout: None, timedout: None, exitcode: 0, output: pull: building file list...
14:37:38 INFO - /sdcard/raptor/profile/minidumps/: 0 files pulled.
14:37:38 INFO - raptor-main removing webext /builds/worker/workspace/build/tests/raptor/raptor/../webext/raptor
14:37:38 INFO - results-handler summarizing raptor test results
14:37:38 INFO - raptor-output error: no raptor test results found for raptor-speedometer-fennec
14:37:38 INFO - raptor-output error: no summarized raptor results found for raptor-speedometer-fennec
14:37:38 INFO - raptor-control-server shutting down control server
14:37:39 INFO - raptor-main removing reverse socket connections
14:37:39 INFO - adb command_output: adb -s HT85S1A02743 wait-for-device reverse --remove-all, timeout: None, timedout: None, exitcode: 0, output:
14:37:39 INFO - raptor-main finished
14:37:39 INFO - raptor-main TEST-UNEXPECTED-FAIL: no raptor test results were found for raptor-speedometer-fennec
14:37:39 ERROR - Return code: 1

Comment 1

[Stephen Donner [:stephend] Not actively reading bugmail]

Taking and looking.

Comment 2

[Intermittent Failures Robot]

4 failures in 2818 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 4

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-nightly: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 3

[Stephen Donner [:stephend] Not actively reading bugmail]

I don't have a Pixel 2, but rather a Moto G5, however running this with --debug-mode yielded this:

04:21:26     INFO -  results-handler received results in RaptorResultsHandler.add
04:21:26     INFO -  127.0.0.1 - - [11/Mar/2019 21:21:26] "POST / HTTP/1.1" 200 -
04:21:26     INFO -  raptor-control-server received webext_status: __raptor_shutdownBrowser
04:21:26     INFO -  raptor-control-server debug-mode enabled, so NOT shutting down the browser
04:21:26     INFO -  adb command_output: adb -s ZY322MQ6ZJ wait-for-device pull /sdcard/raptor/profile/minidumps /var/folders/zc/m_dxr3r564l7m80rynhpr7280000gn/T/tmpgNn8QS/minidumps, timeout: None, timedout: None, exitcode: 0, output: pull: building file list...
04:21:26     INFO -  /sdcard/raptor/profile/minidumps/: 0 files pulled.
04:21:26     INFO -  raptor-main removing webext /Users/stephendonner/moz_source/mozilla-unified/testing/raptor/raptor/../webext/raptor
04:21:26     INFO -  raptor-main * debug-mode enabled - please shutdown the browser manually...
04:21:26    ERROR -  Traceback (most recent call last):
04:21:26     INFO -    File "/Users/stephendonner/moz_source/mozilla-unified/testing/raptor/raptor/raptor.py", line 732, in <module>
04:21:26     INFO -      main()
04:21:26     INFO -    File "/Users/stephendonner/moz_source/mozilla-unified/testing/raptor/raptor/raptor.py", line 701, in main
04:21:26     INFO -      raptor.run_test(next_test, timeout=int(next_test['page_timeout']))
04:21:26     INFO -    File "/Users/stephendonner/moz_source/mozilla-unified/testing/raptor/raptor/raptor.py", line 623, in run_test
04:21:26     INFO -      self.runner.wait(timeout=None)
04:21:26     INFO -  AttributeError: 'RaptorAndroid' object has no attribute 'runner'
04:21:26    ERROR - Return code: 1
04:21:26  WARNING - setting return code to 1
04:21:26     INFO - Killing logcat pid 7563.
04:21:27 CRITICAL - PERFHERDER_DATA was seen 0 times, expected 1.

Attaching full logfile next.

Comment 4

[Stephen Donner [:stephend] Not actively reading bugmail]

Attachment: full-logfile.txt

Are there any other things to try to debug? I'm looking through https://wiki.mozilla.org/Performance_sheriffing/Raptor#Manual_Debugging_on_Firefox_Android too, still.

Comment 5

[Robert Wood [:rwood]]

Thanks Stephen.

I'm guessing this fennec-speedometer intermittent is caused by the same issue in Bug 1533059. We can leave this bug open for now but I'll mark it as depends on Bug 1533059.

As for the error that you discovered in Comment 3

04:21:26 INFO - AttributeError: 'RaptorAndroid' object has no attribute 'runner'

That is a different issue with --debug-mode on Raptor android tests. If you could please file a separate bug for that, that'd be great! Thanks!

Comment 6

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Robert Wood [:rwood] from comment #5)

<snip>

As for the error that you discovered in Comment 3

04:21:26 INFO - AttributeError: 'RaptorAndroid' object has no attribute 'runner'

That is a different issue with --debug-mode on Raptor android tests. If you could please file a separate bug for that, that'd be great! Thanks!

Just filed bug 1534716!

Comment 7

[Intermittent Failures Robot]

3 failures in 2771 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 1
• try: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-nightly: 1
• android-hw-p2-8-0-android-aarch64-nightly: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-03-11&endday=2019-03-17&tree=all

Comment 8

[Intermittent Failures Robot]

2 failures in 2561 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-nightly: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-03-18&endday=2019-03-24&tree=all

Comment 9

[Intermittent Failures Robot]

2 failures in 3910 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-nightly: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-03-25&endday=2019-03-31&tree=all

Comment 10

[Intermittent Failures Robot]

9 failures in 2843 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 5
• mozilla-beta: 3
• mozilla-central: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-nightly: 4
• android-hw-p2-8-0-android-aarch64-nightly: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-04-01&endday=2019-04-07&tree=all

Comment 11

[Intermittent Failures Robot]

10 failures in 3025 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 6
• try: 2
• mozilla-central: 2

Platform breakdown:

• android-em-7-0-x86_64: 1
• android-hw-p2-8-0-android-aarch64-nightly: 4
• android-hw-p2-8-0-arm7-api-16-nightly: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-04-08&endday=2019-04-14&tree=all

Comment 12

[Intermittent Failures Robot]

3 failures in 2673 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 1
• mozilla-central: 2

Platform breakdown:

• android-hw-p2-8-0-android-aarch64-nightly: 2
• android-hw-p2-8-0-arm7-api-16-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-04-15&endday=2019-04-21&tree=all

Comment 13

[Intermittent Failures Robot]

2 failures in 3031 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16-nightly: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1533454&startday=2019-04-22&endday=2019-04-28&tree=all

Comment 14

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup