1524495 - Intermittent raptor-main TEST-UNEXPECTED-FAIL: no raptor test results were found for raptor-unity-webgl-geckoview

Status: RESOLVED FIXED

(Testing :: Raptor, defect, P5)

Description

[Treeherder Bug Filer]

#[markdown(off)]
Filed by: archaeopteryx [at] coole-files.de

http://localhost:5000/logviewer.html#?job_id=225400069&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/SpuRw0gCT_u0_VGg6J_fDQ/runs/0/artifacts/public/logs/live_backing.log

08:09:05 INFO - Activity: org.mozilla.geckoview_example/.GeckoViewActivity
08:09:05 INFO - ThisTime: 448
08:09:05 INFO - TotalTime: 448
08:09:05 INFO - WaitTime: 462
08:09:05 INFO - Complete
08:09:08 INFO - 127.0.0.1 - - [01/Feb/2019 08:09:08] "POST / HTTP/1.1" 200 -
08:09:08 INFO - raptor-control-server received webext_status: raptor runner.js is loaded!
08:09:08 INFO - 127.0.0.1 - - [01/Feb/2019 08:09:08] "GET /raptor-unity-webgl-geckoview.json HTTP/1.1" 200 -
08:09:08 INFO - raptor-control-server reading test settings from raptor-unity-webgl-geckoview.json
08:09:08 INFO - raptor-control-server sent test settings to web ext runner
08:09:08 INFO - 127.0.0.1 - - [01/Feb/2019 08:09:08] "POST / HTTP/1.1" 200 -
08:09:08 INFO - raptor-control-server received webext_status: * pausing 30 seconds to let browser settle... *
08:09:38 INFO - 127.0.0.1 - - [01/Feb/2019 08:09:38] "POST / HTTP/1.1" 200 -
08:09:38 INFO - raptor-control-server received webext_status: running 1 pagecycles of http://127.0.0.1:60169/unity-webgl/index.html?raptor
08:09:39 INFO - 127.0.0.1 - - [01/Feb/2019 08:09:39] "POST / HTTP/1.1" 200 -
08:09:39 INFO - raptor-control-server received webext_status: begin pagecycle 1
08:09:40 INFO - 127.0.0.1 - - [01/Feb/2019 08:09:40] "POST / HTTP/1.1" 200 -
08:09:40 INFO - raptor-control-server received webext_status: update tab 0
08:09:40 INFO - 127.0.0.1 - - [01/Feb/2019 08:09:40] "POST / HTTP/1.1" 200 -
08:09:40 INFO - raptor-control-server received webext_status: test tab updated 0
08:24:32 INFO - raptor-main application timed out after 930 seconds
08:24:33 INFO - adb shell_output: adb -s ZY322LHDDG wait-for-device shell am force-stop org.mozilla.geckoview_example; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
08:24:34 INFO - adb command_output: adb -s ZY322LHDDG wait-for-device pull /sdcard/raptor/profile/minidumps /tmp/tmpFx84NN/minidumps, timeout: None, timedout: None, exitcode: 0, output: pull: building file list...
08:24:34 INFO - /sdcard/raptor/profile/minidumps/: 0 files pulled.
08:24:34 INFO - raptor-main removing webext /builds/worker/workspace/build/tests/raptor/webext/raptor
08:24:34 INFO - results-handler summarizing raptor test results
08:24:34 INFO - raptor-output error: no raptor test results found for raptor-unity-webgl-geckoview
08:24:34 INFO - raptor-output error: no summarized raptor results found for raptor-unity-webgl-geckoview
08:24:34 INFO - raptor-control-server shutting down control server
08:24:34 INFO - raptor-main removing reverse socket connections
08:24:34 INFO - adb command_output: adb -s ZY322LHDDG wait-for-device reverse --remove-all, timeout: None, timedout: None, exitcode: 0, output:
08:24:34 INFO - raptor-main finished
08:24:34 INFO - raptor-main TEST-UNEXPECTED-FAIL: no raptor test results were found for raptor-unity-webgl-geckoview

Comment 1

[Intermittent Failures Robot]

16 failures in 481 pushes (0.033 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• mozilla-inbound: 4

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 15
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-01&endday=2019-02-01&tree=all

Comment 2

[Andreea Pavel [:apavel]]

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=225400069&repo=mozilla-inbound&lineNumber=1062

Bob can this be from Bug 1520130 ?

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&searchStr=android%2C7.0%2Cmotog5%2Copt%2Craptor%2Cperformance%2Ctests%2Con%2Cfirefox%2Cwith%2Ce10s%2Ctest-android-hw-g5-7-0-arm7-api-16%2Fopt-raptor-unity-webgl-geckoview-e10s%2Crap-e10s%28ugl%29&tochange=e565291ee5f6c863a5e247c078f0358142fb0062&fromchange=2fc5715105a265c701027c2a945e1bc1f3f26755&selectedJob=225379628

Comment 3

[Bob Clary [:bc] (inactive)]

Yes. This is the same error we have seen for all raptor tests but now we know which test failed and can track them separately.

Comment 4

[Andreea Pavel [:apavel]]

(In reply to Bob Clary [:bc:] from comment #3)

Yes. This is the same error we have seen for all raptor tests but now we know which test failed and can track them separately.

Thank you for clarifying Bob.

Comment 5

[Intermittent Failures Robot]

25 failures in 2628 pushes (0.01 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 18
• mozilla-central: 1
• mozilla-inbound: 6

Platform breakdown:

• linux64-qr: 1
• android-hw-g5-7-0-arm7-api-16: 22
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-01-28&endday=2019-02-03&tree=all

Comment 6

[Intermittent Failures Robot]

17 failures in 379 pushes (0.045 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-inbound: 7

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 15
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-04&endday=2019-02-04&tree=all

Comment 7

[Intermittent Failures Robot]

21 failures in 478 pushes (0.044 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 2
• autoland: 16
• mozilla-inbound: 2
• try: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 18
• android-hw-p2-8-0-arm7-api-16: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-06&endday=2019-02-06&tree=all

Comment 8

[Intermittent Failures Robot]

17 failures in 486 pushes (0.035 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 10
• mozilla-inbound: 6
• mozilla-central: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 3
• android-hw-g5-7-0-arm7-api-16: 14

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-07&endday=2019-02-07&tree=all

Comment 9

[Intermittent Failures Robot]

15 failures in 550 pushes (0.027 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 9
• mozilla-inbound: 5
• mozilla-central: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 13
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-08&endday=2019-02-08&tree=all

Comment 10

[Intermittent Failures Robot]

96 failures in 2658 pushes (0.036 failures/push) were associated with this bug in the last 7 days.

This is the #14 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 56
• try: 1
• mozilla-inbound: 34
• mozilla-central: 5

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 14
• android-hw-g5-7-0-arm7-api-16: 82

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-04&endday=2019-02-10&tree=all

Comment 11

[Natalia Csoregi [:nataliaCs]]

Update: there have been 88 failures within the last 7 days on Android 8.0 Pixel2 opt and Android 7.0 MotoG5 opt

Recent failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=227762796&repo=mozilla-inbound&lineNumber=1064

22:41:47 INFO - 127.0.0.1 - - [11/Feb/2019 22:41:47] "POST / HTTP/1.1" 200 -
22:41:47 INFO - raptor-control-server received webext_status: raptor runner.js is loaded!
22:41:47 INFO - 127.0.0.1 - - [11/Feb/2019 22:41:47] "GET /raptor-unity-webgl-geckoview.json HTTP/1.1" 200 -
22:41:47 INFO - raptor-control-server reading test settings from raptor-unity-webgl-geckoview.json
22:41:47 INFO - raptor-control-server sent test settings to web ext runner
22:41:47 INFO - 127.0.0.1 - - [11/Feb/2019 22:41:47] "POST / HTTP/1.1" 200 -
22:41:47 INFO - raptor-control-server received webext_status: * pausing 30 seconds to let browser settle... *
22:42:17 INFO - 127.0.0.1 - - [11/Feb/2019 22:42:17] "POST / HTTP/1.1" 200 -
22:42:17 INFO - raptor-control-server received webext_status: running 1 pagecycles of http://127.0.0.1:34196/unity-webgl/index.html?raptor
22:42:18 INFO - 127.0.0.1 - - [11/Feb/2019 22:42:18] "POST / HTTP/1.1" 200 -
22:42:18 INFO - raptor-control-server received webext_status: begin pagecycle 1
22:42:19 INFO - 127.0.0.1 - - [11/Feb/2019 22:42:19] "POST / HTTP/1.1" 200 -
22:42:19 INFO - raptor-control-server received webext_status: update tab 0
22:42:19 INFO - 127.0.0.1 - - [11/Feb/2019 22:42:19] "POST / HTTP/1.1" 200 -
22:42:19 INFO - raptor-control-server received webext_status: test tab updated 0
22:57:13 INFO - raptor-main application timed out after 930 seconds
22:57:13 INFO - adb shell_output: adb -s HT85S1A02559 wait-for-device shell am force-stop org.mozilla.geckoview_example; echo adb_returncode=$?, timeout: None, root: False, timedout: None, exitcode: 0, output:
22:57:14 INFO - adb command_output: adb -s HT85S1A02559 wait-for-device pull /sdcard/raptor/profile/minidumps /tmp/tmp4rIl6J/minidumps, timeout: None, timedout: None, exitcode: 0, output: pull: building file list...
22:57:14 INFO - /sdcard/raptor/profile/minidumps/: 0 files pulled.
22:57:14 INFO - raptor-main removing webext /builds/worker/workspace/build/tests/raptor/webext/raptor
22:57:14 INFO - results-handler summarizing raptor test results
22:57:14 INFO - raptor-output error: no raptor test results found for raptor-unity-webgl-geckoview
22:57:14 INFO - raptor-output error: no summarized raptor results found for raptor-unity-webgl-geckoview
22:57:14 INFO - raptor-control-server shutting down control server
22:57:14 INFO - raptor-main removing reverse socket connections
22:57:14 INFO - adb command_output: adb -s HT85S1A02559 wait-for-device reverse --remove-all, timeout: None, timedout: None, exitcode: 0, output:
22:57:14 INFO - raptor-main finished
22:57:14 INFO - raptor-main TEST-UNEXPECTED-FAIL: no raptor test results were found for raptor-unity-webgl-geckoview
22:57:15 ERROR - Return code: 1
22:57:15 WARNING - setting return code to 1
22:57:15 INFO - Killing logcat pid 431.
22:57:15 CRITICAL - PERFHERDER_DATA was seen 0 times, expected 1.
22:57:15 INFO - copying raptor results to upload dir:
22:57:15 INFO - /builds/worker/workspace/build/blobber_upload_dir/perfherder-data.json
22:57:15 INFO - copying raptor results from /builds/worker/workspace/build/raptor.json to /builds/worker/workspace/build/blobber_upload_dir/perfherder-data.json
22:57:15 CRITICAL - Error copying results /builds/worker/workspace/build/raptor.json to upload dir /builds/worker/workspace/build/blobber_upload_dir/perfherder-data.json
22:57:15 INFO - [Errno 2] No such file or directory: u'/builds/worker/workspace/build/raptor.json'

Comment 12

[Intermittent Failures Robot]

22 failures in 564 pushes (0.039 failures/push) were associated with this bug yesterday.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• try: 1
• mozilla-central: 2
• autoland: 13
• mozilla-beta: 1
• mozilla-inbound: 5

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 1
• android-hw-g5-7-0-arm7-api-16: 21

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-12&endday=2019-02-12&tree=all

Comment 13

[Intermittent Failures Robot]

21 failures in 471 pushes (0.045 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-central: 1
• mozilla-inbound: 7
• autoland: 13

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 21

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-14&endday=2019-02-14&tree=all

Comment 14

[Intermittent Failures Robot]

19 failures in 588 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 13
• mozilla-central: 3
• mozilla-inbound: 3

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 17
• android-hw-p2-8-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-15&endday=2019-02-15&tree=all

Comment 15

[Intermittent Failures Robot]

85 failures in 2832 pushes (0.03 failures/push) were associated with this bug in the last 7 days.

This is the #11 most frequent failure this week.

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. **

Repository breakdown:

• autoland: 55
• mozilla-central: 6
• try: 2
• mozilla-beta: 1
• mozilla-inbound: 21

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 81
• android-hw-p2-8-0-arm7-api-16: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-11&endday=2019-02-17&tree=all

Comment 16

[Dorel Luca [:dluca]]

This bug failed 71 times in the last 7 days. Occurs on Android 8.0 Pixel2 opt and Android 7.0 MotoG5 opt.

Recent log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=229357007&repo=autoland&lineNumber=1069

rwood: Can you please take a look at this bug?

Comment 17

[Intermittent Failures Robot]

16 failures in 425 pushes (0.038 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 11
• mozilla-inbound: 4
• mozilla-central: 1

Platform breakdown:

• android-hw-p2-8-0-arm7-api-16: 1
• android-hw-g5-7-0-arm7-api-16: 15

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-20&endday=2019-02-20&tree=all

Comment 18

[Intermittent Failures Robot]

18 failures in 597 pushes (0.03 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-inbound: 2
• autoland: 13
• mozilla-central: 2
• try: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 17
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-21&endday=2019-02-21&tree=all

Comment 19

[Intermittent Failures Robot]

16 failures in 500 pushes (0.032 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 12
• mozilla-inbound: 4

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 16

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-22&endday=2019-02-22&tree=all

Comment 20

[Narcis Beleuzu [:NarcisB]]

Attachment: Bug 1524495 - Temporarily disable raptor-unity-webgl-geckoview due to frequent failures

Comment 21

[Intermittent Failures Robot]

75 failures in 2602 pushes (0.029 failures/push) were associated with this bug in the last 7 days.

This is the #13 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 49
• mozilla-inbound: 17
• mozilla-central: 5
• try: 3
• mozilla-beta: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 67
• android-hw-p2-8-0-arm7-api-16: 8

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-18&endday=2019-02-24&tree=all

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9b617c97285a
Temporarily disable raptor-unity-webgl-geckoview due to frequent failures r=jmaher

Comment 23

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/984c1d9f8a44
Backed out changeset 9b617c97285a for Py2 failure

Comment 24

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e5fc352e6d65
Temporarily disable raptor-unity-webgl-geckoview due to frequent failures r=jmaher

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/4f37d111e4a9
Backed out changeset e5fc352e6d65 for Raptor performance tests failures.
https://hg.mozilla.org/integration/autoland/rev/2e473aa11c19
Temporarily disable raptor-unity-webgl-geckoview due to frequent failures. r=jmaher

Comment 26

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/e5fc352e6d65

Comment 27

[Narcis Beleuzu [:NarcisB]]

Last disable push did his job but triggered a perma Android 7.0 MotoG5 opt - ugl failure.
Log link: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=230581981&repo=autoland&lineNumber=894

I`m not sure how we can properly disable it.
:rwood , can you please take a look?

Comment 28

[Bob Clary [:bc] (inactive)]

If we still want to disable this, I believe doing so in raptor.yml is the right place.
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/raptor.yml

Comment 29

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/2e473aa11c19

Comment 30

[Cristina Coroiu [:ccoroiu]]

This is permafailing tier2:
https://treeherder.mozilla.org/#/jobs?repo=autoland&fromchange=d819234834aad12e3388ceb0b09dfb6c8cdd3da9&selectedJob=230798330&searchStr=android%2C7.0%2Cmotog5%2Copt%2Craptor%2Cperformance%2Ctests%2Con%2Cfirefox%2Cwith%2Ce10s%2Ctest-android-hw-g5-7-0-arm7-api-16%2Fopt-raptor-unity-webgl-geckoview-e10s%2Crap-e10s%28ugl%29

Comment 31

[Intermittent Failures Robot]

23 failures in 537 pushes (0.043 failures/push) were associated with this bug yesterday.

Repository breakdown:

• autoland: 11
• mozilla-inbound: 12

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 23

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-27&endday=2019-02-27&tree=all

Comment 32

[Cosmin Sabou [:CosminS]]

Had two tries on fixing this on try but to no avail:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=5249fc7eb727e0a2461171eecb3268c95a3e8871&selectedJob=230963976

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6e899fa6dc062670f62d12eef5021cd0471e4c48&selectedJob=230959091

The actual fail I think complains about not finding the test anymore since has been disabled.

07:37:52 INFO - Calling ['/builds/worker/workspace/build/venv/bin/python', u'/builds/worker/workspace/build/tests/raptor/raptor/raptor.py', u'--test', 'raptor-unity-webgl', u'--binary', 'org.mozilla.geckoview_example', u'--app', 'geckoview', u'--symbolsPath', 'https://queue.taskcluster.net/v1/task/Fvq-V7AKT5OvonpL8SLjdA/artifacts/public/build/target.crashreporter-symbols.zip', u'--log-tbpl-level=debug'] with output_timeout 3600
07:37:53 INFO - raptor-main raptor-start
07:37:53 INFO - raptor-main received command line arguments: Namespace(activity='GeckoViewActivity', app='geckoview', binary='org.mozilla.geckoview_example', debug_mode=False, gecko_profile=False, gecko_profile_entries=None, gecko_profile_interval=None, host='127.0.0.1', is_release_build=False, log_errorsummary=None, log_grouped=None, log_html=None, log_mach=None, log_mach_buffer=None, log_mach_level=None, log_mach_screenshot=None, log_mach_verbose=None, log_raw=None, log_raw_level=None, log_tbpl=None, log_tbpl_buffer=None, log_tbpl_compact=None, log_tbpl_level='debug', log_unittest=None, log_xunit=None, obj_path=None, page_cycles=None, page_timeout=None, power_test=False, run_local=False, symbols_path='https://queue.taskcluster.net/v1/task/Fvq-V7AKT5OvonpL8SLjdA/artifacts/public/build/target.crashreporter-symbols.zip', test='raptor-unity-webgl')
07:37:53 INFO - raptor-manifest /builds/worker/workspace/build/tests/raptor/raptor/raptor.ini
07:37:53 INFO - raptor-manifest abort: specified test name doesn't exist
07:37:53 INFO - raptor-main abort: no tests found

raptor.ini contains the whole chunk of those test, so no possibility of disabling this from there: https://searchfox.org/mozilla-central/source/testing/raptor/raptor/raptor.ini#26

Looking at https://searchfox.org/mozilla-central/source/taskcluster/ci/test/raptor.yml#656 disabling this directly from that file seems a bit too much for me. Hopefully Robert could take a look at this.

Comment 33

[Robert Wood [:rwood]]

(In reply to Cosmin Sabou [:CosminS] from comment #32)

Looking at https://searchfox.org/mozilla-central/source/taskcluster/ci/test/raptor.yml#656 disabling this directly from that file seems a bit too much for me. Hopefully Robert could take a look at this.

Thanks Cosmin yes using 'diabled =' in a test INI in production is only good for subtests (i.e. tp6* suites) as there are other subtests in the suite that are still found/run. Disabling an entire test at the test level / just one test has to be done at the taskcluster config level as you noted raptor.yml. I'll make a patch now.

Comment 34

[Robert Wood [:rwood]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0b5cd585f1577db4c4032f7e0bb7556fcc4bc4b1

Comment 35

[Robert Wood [:rwood]]

Attachment: Bug 1524495 - Disble Raptor ugl geckoview job b/c of permafail; r?davehunt,stephendonner

Comment 36

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1a272d807e84
Disble Raptor ugl geckoview job b/c of permafail; r=davehunt

Comment 37

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/1a272d807e84

Comment 38

[Intermittent Failures Robot]

34 failures in 504 pushes (0.067 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-inbound: 21
• autoland: 11
• mozilla-central: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 33
• android-hw-p2-8-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-28&endday=2019-02-28&tree=all

Comment 39

[Cosmin Sabou [:CosminS]]

This was not actually fixed but disabled @comment 36 so setting the flags accordingly.

Comment 40

[Intermittent Failures Robot]

80 failures in 2981 pushes (0.027 failures/push) were associated with this bug in the last 7 days.

This is the #15 most frequent failure this week.

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-inbound: 46
• autoland: 31
• mozilla-central: 3

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 78
• android-hw-p2-8-0-arm7-api-16: 1
• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-02-25&endday=2019-03-03&tree=all

Comment 41

[Intermittent Failures Robot]

2 failures in 2818 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-03-04&endday=2019-03-10&tree=all

Comment 42

[Intermittent Failures Robot]

1 failures in 3910 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• android-hw-g5-7-0-arm7-api-16: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1524495&startday=2019-03-25&endday=2019-03-31&tree=all