1495002 - Permafailure ccov tier2 Wd3 TEST-UNEXPECTED-TIMEOUT | /webdriver/tests/[+/- folder]/test_name| expected OK

Status: RESOLVED FIXED

(Remote Protocol :: Marionette, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: aciure [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=202218864&repo=autoland

https://queue.taskcluster.net/v1/task/fmsgI2FIRq-tuJQrX40qeg/runs/0/artifacts/public/logs/live_backing.log

[task 2018-09-28T13:32:41.958Z] 13:32:41     INFO - PID 26971 | [Parent 6361, Gecko_IOThread] WARNING: pipe error (92): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 356
[task 2018-09-28T13:32:41.965Z] 13:32:41     INFO - PID 26971 | [Parent 6361, Gecko_IOThread] WARNING: pipe error (91): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 356
[task 2018-09-28T13:32:42.062Z] 13:32:42     INFO - PID 26971 | 1538141562052	Marionette	DEBUG	Received observer notification xpcom-will-shutdown
[task 2018-09-28T13:32:42.063Z] 13:32:42     INFO - PID 26971 | 1538141562052	Marionette	DEBUG	Resetting recommended pref security.turn_off_all_security_so_that_viruses_can_take_over_this_computer
[task 2018-09-28T13:32:42.064Z] 13:32:42     INFO - PID 26971 | 1538141562052	Marionette	DEBUG	Resetting recommended pref apz.content_response_timeout
[task 2018-09-28T13:32:42.065Z] 13:32:42     INFO - PID 26971 | 1538141562053	Marionette	DEBUG	Resetting recommended pref browser.download.panel.shown
[task 2018-09-28T13:32:42.065Z] 13:32:42     INFO - PID 26971 | 1538141562053	Marionette	DEBUG	Resetting recommended pref browser.pagethumbnails.capturing_disabled
[task 2018-09-28T13:32:42.066Z] 13:32:42     INFO - PID 26971 | 1538141562053	Marionette	DEBUG	Resetting recommended pref browser.search.update
[task 2018-09-28T13:32:42.068Z] 13:32:42     INFO - PID 26971 | 1538141562053	Marionette	DEBUG	Resetting recommended pref toolkit.cosmeticAnimations.enabled
[task 2018-09-28T13:32:42.069Z] 13:32:42     INFO - PID 26971 | 1538141562053	Marionette	DEBUG	Resetting recommended pref browser.tabs.disableBackgroundZombification
[task 2018-09-28T13:32:42.071Z] 13:32:42     INFO - PID 26971 | 1538141562053	Marionette	DEBUG	Resetting recommended pref browser.tabs.warnOnCloseOtherTabs
[task 2018-09-28T13:32:42.075Z] 13:32:42     INFO - PID 26971 | 1538141562054	Marionette	DEBUG	Resetting recommended pref browser.tabs.warnOnOpen
[task 2018-09-28T13:32:42.076Z] 13:32:42     INFO - PID 26971 | 1538141562054	Marionette	DEBUG	Resetting recommended pref browser.usedOnWindows10.introURL
[task 2018-09-28T13:32:42.076Z] 13:32:42     INFO - PID 26971 | 1538141562054	Marionette	DEBUG	Resetting recommended pref browser.urlbar.suggest.searches
[task 2018-09-28T13:32:42.077Z] 13:32:42     INFO - PID 26971 | 1538141562054	Marionette	DEBUG	Resetting recommended pref datareporting.policy.dataSubmissionPolicyAccepted
[task 2018-09-28T13:32:42.079Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref dom.disable_beforeunload
[task 2018-09-28T13:32:42.080Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref dom.disable_open_during_load
[task 2018-09-28T13:32:42.081Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref dom.file.createInChild
[task 2018-09-28T13:32:42.082Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref dom.max_chrome_script_run_time
[task 2018-09-28T13:32:42.088Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref dom.max_script_run_time
[task 2018-09-28T13:32:42.091Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref extensions.getAddons.cache.enabled
[task 2018-09-28T13:32:42.092Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref extensions.webservice.discoverURL
[task 2018-09-28T13:32:42.094Z] 13:32:42     INFO - PID 26971 | 1538141562055	Marionette	DEBUG	Resetting recommended pref network.http.prompt-temp-redirect
[task 2018-09-28T13:32:42.096Z] 13:32:42     INFO - PID 26971 | 1538141562056	Marionette	DEBUG	Resetting recommended pref network.http.speculative-parallel-limit
[task 2018-09-28T13:32:42.097Z] 13:32:42     INFO - PID 26971 | 1538141562056	Marionette	DEBUG	Resetting recommended pref security.fileuri.strict_origin_policy
[task 2018-09-28T13:32:42.097Z] 13:32:42     INFO - PID 26971 | 1538141562056	Marionette	DEBUG	Resetting recommended pref security.notification_enable_delay
[task 2018-09-28T13:32:42.098Z] 13:32:42     INFO - PID 26971 | 1538141562056	Marionette	DEBUG	Resetting recommended pref signon.autofillForms
[task 2018-09-28T13:32:42.099Z] 13:32:42     INFO - PID 26971 | 1538141562056	Marionette	DEBUG	Resetting recommended pref signon.rememberSignons
[task 2018-09-28T13:32:42.100Z] 13:32:42     INFO - PID 26971 | 1538141562056	Marionette	DEBUG	Remote service is inactive
[task 2018-09-28T13:33:11.712Z] 13:33:11     INFO - TEST-UNEXPECTED-TIMEOUT | /webdriver/tests/new_session/timeouts.py | expected OK
[task 2018-09-28T13:33:11.712Z] 13:33:11     INFO - TEST-INFO took 30002ms
[task 2018-09-28T13:33:11.776Z] 13:33:11  WARNING - u'runner_teardown': ()
[task 2018-09-28T13:33:11.777Z] 13:33:11     INFO - Starting runner
[task 2018-09-28T13:33:11.806Z] 13:33:11     INFO - PID 6578 | 1538141591800	webdriver::httpapi	DEBUG	Creating routes
[task 2018-09-28T13:33:11.807Z] 13:33:11     INFO - WebDriver HTTP server listening at http://127.0.0.1:4445/

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Hang during shutdown of Firefox.

Comment 2

[Intermittent Failures Robot]

2 failures in 2773 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2018-09-24&endday=2018-09-30&tree=all

Comment 3

[Intermittent Failures Robot]

1 failures in 2978 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2018-10-29&endday=2018-11-04&tree=all

Comment 4

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 5

[Andrei Ciure[:aciure]]

New occurrence:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=testfailed%2Cbusted%2Cexception&classifiedState=unclassified&group_state=expanded&selectedJob=217123465&searchStr=windows%2C10%2Cx64%2Cdebug%2Cweb%2Cplatform%2Ctests%2Cwith%2Ce10s%2Ctest-windows10-64%2Fdebug-web-platform-tests-wdspec-e10s-2%2Cw-e10s%28wd2%29

Comment 6

[Intermittent Failures Robot]

1 failures in 2485 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2018-12-10&endday=2018-12-16&tree=all

Comment 7

[Firefox Bug Husbandry Bot]

https://wiki.mozilla.org/Bug_Triage#Intermittent_Test_Failure_Cleanup

Comment 8

[Razvan Maries]

New occurrences: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=220710845&repo=mozilla-central&lineNumber=73713

Comment 9

[Cristina Coroiu [:ccoroiu]]

This started to perma fail with this merge: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=windows%2C10%2Cx64%2Cccov%2Cdebug%2Cweb%2Cplatform%2Ctests%2Cwith%2Ce10s%2Ctest-windows10-64-ccov%2Fdebug-web-platform-tests-wdspec-e10s-4%2Cw-e10s%28wd4%29&revision=1f7a8905fa7cf88143161cb9c65ca69cb1daec65&selectedJob=220646216

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Looks like this is like a rat tail. Once we skip a wdspec test for ccov the next in the row starts to fail. This has been most likely caused by bug 1507121.

Maybe Christina can have a look again, and we might have to skip them all?

Comment 11

[Cristina Coroiu [:ccoroiu]]

Attachment: Disable timeouts.py on win ccov for frequent failures

Created a patch to disable timeouts.py

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Please trigger a try build for ccov tests to verify that we don't trigger another follow-up failure with disabling this test. As best retrigger the Wd jobs multiple times.

Comment 13

[Cristina Coroiu [:ccoroiu]]

Try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=bd05cfe39f9285cd5d19f4345e0d89ebb405016c&group_state=expanded

Comment 14

[Intermittent Failures Robot]

41 failures in 2627 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #25 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 41

Platform breakdown:

• linux64-qr: 1
• windows10-64-ccov: 40

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-01-07&endday=2019-01-13&tree=all

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/39346b9630de
disable timeouts.py on win ccov for frequent failures. r=whimboo

Comment 16

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/39346b9630de

Comment 17

[Intermittent Failures Robot]

17 failures in 2378 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 17

Platform breakdown:

• windows10-64-ccov: 16
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-01-14&endday=2019-01-20&tree=all

Comment 18

[Intermittent Failures Robot]

12 failures in 2832 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• mozilla-central: 11

Platform breakdown:

• linux64-qr: 2
• windows10-64-ccov: 10

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-02-11&endday=2019-02-17&tree=all

Comment 19

[Intermittent Failures Robot]

5 failures in 2602 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 5

Platform breakdown:

• windows10-64-ccov: 4
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-02-18&endday=2019-02-24&tree=all

Comment 20

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The test got accidentally unskipped for ccov builds, and has been marked as skipped again via https://hg.mozilla.org/mozilla-central/rev/ab359d0369df.

Comment 21

[Intermittent Failures Robot]

13 failures in 4439 pushes (0.003 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 8
• autoland: 3
• mozilla-inbound: 1
• try: 1

Platform breakdown:

• linux64-ccov: 7
• linux64-shippable: 5
• linux64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-07-29&endday=2019-08-04&tree=all

Comment 22

[Razvan Maries]

Tier 1 occurrence: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=260691369&repo=mozilla-central&lineNumber=101153

Comment 23

[Intermittent Failures Robot]

33 failures in 4666 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-central: 25
• autoland: 3
• try: 4
• mozilla-inbound: 1

Platform breakdown:

• linux64-ccov: 14
• linux64-shippable: 8
• linux64-shippable-qr: 5
• windows10-64-ccov: 1
• linux64-qr: 3
• linux64-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-08-05&endday=2019-08-11&tree=all

Comment 24

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Why the change from perma to intermittent? This test might still perma fail if it would be enabled.

Comment 25

[Alexandru Michis [:malexandru]]

Seeing as this bug was intermittent when it was first created https://bugzilla.mozilla.org/show_bug.cgi?id=1495002#c9, and then the summary was changed to reflect that it became a permafail, I'd say it isn't worth it to create a new bug for the intermittent variant.

Also disable has been done 6 months ago, so my guess would be that it's only going to be re-enabled after fixing the issue.

Comment 26

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to Alexandru Michis [:malexandru] from comment #25)

Seeing as this bug was intermittent when it was first created https://bugzilla.mozilla.org/show_bug.cgi?id=1495002#c9, and then the summary was changed to reflect that it became a permafail, I'd say it isn't worth it to create a new bug for the intermittent variant.

Also disable has been done 6 months ago, so my guess would be that it's only going to be re-enabled after fixing the issue.

Please note that this is a new regression and started happening for this test on Aug 2nd, and that only on the Linux platform. Everyone filed those failures against this bug without noticing that the platform is totally different. A check for the disable patch would have been enough. So please file a new bug because it is totally different, and would need to be investigated.

Comment 27

[Intermittent Failures Robot]

35 failures in 4839 pushes (0.007 failures/push) were associated with this bug in the last 7 days.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• autoland: 7
• try: 5
• mozilla-central: 20
• mozilla-inbound: 3

Platform breakdown:

• linux64-shippable: 15
• linux64-shippable-qr: 5
• linux64-ccov: 9
• linux64-devedition: 2
• linux64: 2
• linux64-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-08-12&endday=2019-08-18&tree=all

Comment 28

[Alexandru Michis [:malexandru]]

Understood, I've made a new bug for the failures unrelated to win ccov: https://bugzilla.mozilla.org/show_bug.cgi?id=1574837 and reclassified these failures against it.

Comment 29

[Intermittent Failures Robot]

3 failures in 5047 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3

Platform breakdown:

• linux64-ccov: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-08-26&endday=2019-09-01&tree=all

Comment 30

[Intermittent Failures Robot]

2 failures in 4408 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 2

Platform breakdown:

• linux64-ccov: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-09-02&endday=2019-09-08&tree=all

Comment 31

[Intermittent Failures Robot]

6 failures in 4637 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 6

Platform breakdown:

• linux64-ccov: 6

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-09-09&endday=2019-09-15&tree=all

Comment 32

[Intermittent Failures Robot]

4 failures in 4350 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 4

Platform breakdown:

• linux64-ccov: 4

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-09-16&endday=2019-09-22&tree=all

Comment 33

[Intermittent Failures Robot]

5 failures in 4530 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 5

Platform breakdown:

• linux64-ccov: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-09-23&endday=2019-09-29&tree=all

Comment 34

[Intermittent Failures Robot]

5 failures in 4292 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 5

Platform breakdown:

• linux64-ccov: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-09-30&endday=2019-10-06&tree=all

Comment 35

[Intermittent Failures Robot]

7 failures in 3801 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 7

Platform breakdown:

• linux64-ccov: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-10-07&endday=2019-10-13&tree=all

Comment 36

[Intermittent Failures Robot]

5 failures in 3964 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 5

Platform breakdown:

• linux64-ccov: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-10-14&endday=2019-10-20&tree=all

Comment 37

[Andreea Pavel [:apavel]]

Attachment: Bug 1495002 - disable permissions/set.py on win ccov debug for frequent failures r=whimboo

Comment 38

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3538810d52da
disable permissions/set.py on win ccov debug for frequent failures r=whimboo

Comment 39

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/3538810d52da

Comment 42

[Andreea Pavel [:apavel]]

Attachment: Bug 1495002 - disable new_session/create_alwaysMatch.py and new_session/timeouts.py on linux ccov debug for frequent failures r=whimboo

Comment 43

[Andreea Pavel [:apavel]]

Henrik, I closed the above bugs as duplicates based on https://bugzilla.mozilla.org/show_bug.cgi?id=1526678#c27

Comment 44

[Intermittent Failures Robot]

23 failures in 4182 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 23

Platform breakdown:

• windows10-64-ccov: 17
• linux64-ccov: 5
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-10-21&endday=2019-10-27&tree=all

Comment 45

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Carrying over dependency on bug 1577713.

Comment 46

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5aa41cf3904b
disable new_session/create_alwaysMatch.py and new_session/timeouts.py on linux ccov debug for frequent failures r=whimboo

Comment 47

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/5aa41cf3904b

Comment 49

[Intermittent Failures Robot]

3 failures in 4314 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1
• mozilla-release: 1
• autoland: 1

Platform breakdown:

• linux64-ccov: 1
• linux64-shippable: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-10-28&endday=2019-11-03&tree=all

Comment 50

[Intermittent Failures Robot]

1 failures in 5444 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-release: 1

Platform breakdown:

• linux64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-11-18&endday=2019-11-24&tree=all

Comment 51

[Intermittent Failures Robot]

1 failures in 4331 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• macosx1014-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-11-25&endday=2019-12-01&tree=all

Comment 52

[Intermittent Failures Robot]

1 failures in 4867 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-12-09&endday=2019-12-15&tree=all

Comment 53

[Intermittent Failures Robot]

2 failures in 4582 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 2

Platform breakdown:

• windows10-64-shippable: 1
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-12-16&endday=2019-12-22&tree=all

Comment 54

[Intermittent Failures Robot]

1 failures in 1365 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-12-23&endday=2019-12-29&tree=all

Comment 55

[Intermittent Failures Robot]

2 failures in 1862 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 2

Platform breakdown:

• windows7-32-shippable: 1
• macosx1014-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2019-12-30&endday=2020-01-05&tree=all

Comment 56

[Marco Castelluccio [:marco]]

Attachment: Bug 1495002 - Don't disable create_alwaysMatch.py on Linux ccov debug, since this build doesn't exist anymore. r=whimboo

Comment 57

[Marco Castelluccio [:marco]]

Attachment: Bug 1495002 - Don't disable timeouts.py on Linux ccov debug, since this build doesn't exist anymore. r=whimboo

Depends on D59244

Comment 58

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2e0027ec4127
Don't disable create_alwaysMatch.py on Linux ccov debug, since this build doesn't exist anymore. r=whimboo
https://hg.mozilla.org/integration/autoland/rev/0c2f333260eb
Don't disable timeouts.py on Linux ccov debug, since this build doesn't exist anymore. r=whimboo

Comment 59

[Marco Castelluccio [:marco]]

It hasn't been happening for a while since we switched to an opt build.

Comment 60

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/2e0027ec4127
https://hg.mozilla.org/mozilla-central/rev/0c2f333260eb

Comment 61

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/2e0027ec4127
https://hg.mozilla.org/mozilla-central/rev/0c2f333260eb

Comment 62

[Intermittent Failures Robot]

6 failures in 4472 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 4
• autoland: 1
• try: 1

Platform breakdown:

• windows7-32-shippable: 4
• windows10-64-shippable: 1
• macosx1014-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-01-06&endday=2020-01-12&tree=all

Comment 63

[Cosmin Sabou [:CosminS]]

Marco, this is still happening on windows ccov opt jobs https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=windows%2C10%2Cx64%2Cccov%2Copt%2Cweb%2Cplatform%2Ctests%2Ctest-windows10-64-ccov%2Fopt-web-platform-tests-wdspec-e10s-3%2Cw%28wd3%29&fromchange=1536cf66a302811c1e302118e0d356c6c7545fab&tochange=2ee6c4c250523ed74cc95c155abc0fd802113d2e&selectedJob=284995065

Should we remove the |debug| part from https://searchfox.org/mozilla-central/source/testing/web-platform/meta/webdriver/tests/permissions/set.py.ini#3 ?

Comment 64

[Marco Castelluccio [:marco]]

Attachment: Bug 1495002 - Disable webdriver/tests/permissions/set.py on Windows coverage build (either opt or debug). r=whimboo

Comment 65

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3f799b7f6517
Disable webdriver/tests/permissions/set.py on Windows coverage build (either opt or debug). r=whimboo

Comment 66

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/3f799b7f6517

Comment 67

[Intermittent Failures Robot]

6 failures in 3773 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• mozilla-central: 4
• autoland: 1

Platform breakdown:

• windows7-32-shippable: 1
• windows10-64-ccov: 4
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-01-13&endday=2020-01-19&tree=all

Comment 68

[Henrik Skupin [:whimboo][⌚️UTC+1]]

webdriver/tests/minimize_window/user_prompts.py is also failing permanently on Windows and as such need to be disabled.

Comment 69

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: Bug 1495002 - [wdspec] Disable minimize_window/user_prompts.py for win ccov builds.

Comment 70

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/673d14980ca0

Comment 71

[Intermittent Failures Robot]

2 failures in 4004 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• mozilla-beta: 1

Platform breakdown:

• windows7-32-shippable: 1
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-01-20&endday=2020-01-26&tree=all

Comment 72

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Marco, should we be ok now to unskip all of those tests now?

Comment 73

[Marco Castelluccio [:marco]]

Yeah, we could try and if they still fail so often we'll disable them again.

Comment 74

[Marco Castelluccio [:marco]]

Same with bug 1608082, bug 1608084, bug 1608086 and bug 1608956.

Comment 75

[Andreea Pavel [:apavel]]

Hi Henrik, want us to unskip the tests or will you do it?

Comment 76

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Andreea, if one of you has the time to do that I would kinda appreciate. Thanks!

Comment 77

[Andreea Pavel [:apavel]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] from comment #76)

Andreea, if one of you has the time to do that I would kinda appreciate. Thanks!

Sure, we'll do it. Thank you.

Comment 78

[Andreea Pavel [:apavel]]

Attachment: Bug 1495002 - [wdspec] Re-enable all tests for ccov except permissions/set.py.ini which still perma-fails.

Comment 79

[Intermittent Failures Robot]

3 failures in 3364 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1
• autoland: 1
• mozilla-release: 1

Platform breakdown:

• windows7-32-devedition: 1
• linux64-qr: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-02-03&endday=2020-02-09&tree=all

Comment 81

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6436566bc9a1
[wdspec] Re-enable all tests for ccov except permissions/set.py.ini which still perma-fails. r=whimboo

Comment 82

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/6436566bc9a1

Comment 83

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We still have perma-failures for alwaysMatch and timeout on Windows 10:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunning%2Cpending%2Crunnable&tier=1%2C2%2C3&searchStr=wd%2Cccov

I will have a look why we have such hangs during startup now, so that we can hopefully fix this problem (bug 1382162). As such I will push some try builds for investigating the issue.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8bf3deae3915870cee59704cfdd2db070467389f

Andreea, if you want to skip them again feel free to create a patch.

Comment 84

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The job failed again:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=288672033&repo=try

[task 2020-02-13T09:54:53.196Z] 09:54:53     INFO - TEST-START | /webdriver/tests/new_session/timeouts.py
[..]
[task 2020-02-13T09:55:48.203Z] 09:55:48     INFO - TEST-UNEXPECTED-TIMEOUT | /webdriver/tests/new_session/timeouts.py | expected OK

This test doesn't use the meta tag for a longer timeout, so it should be added given that we start multiple instances of Firefox.

[task 2020-02-13T09:44:36.974Z] 09:44:36     INFO - TEST-START | /webdriver/tests/new_session/create_alwaysMatch.py
[..]
[task 2020-02-13T09:50:41.985Z] 09:50:41     INFO - TEST-UNEXPECTED-TIMEOUT | /webdriver/tests/new_session/create_alwaysMatch.py | expected OK

This test runs about 6 minutes because it starts/stops even more Firefox instances. Seeing that it takes about 11s for a safe shutdown of Firefox, and 8s for startup we most likely should split this test into multiple files.

Marco, I assume those startup/shutdown times are normal?

Comment 85

[Andreea Pavel [:apavel]]

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+1] (away 02/17 - 02/21) from comment #83)

We still have perma-failures for alwaysMatch and timeout on Windows 10:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunning%2Cpending%2Crunnable&tier=1%2C2%2C3&searchStr=wd%2Cccov

I will have a look why we have such hangs during startup now, so that we can hopefully fix this problem (bug 1382162). As such I will push some try builds for investigating the issue.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8bf3deae3915870cee59704cfdd2db070467389f

Andreea, if you want to skip them again feel free to create a patch.

Sure, will do it in a couple of minutes.

Comment 86

[Andreea Pavel [:apavel]]

Attachment: Bug 1495002 - disable create_alwaysMatch.py and timeouts.py in win ccov r=whimboo

Comment 87

[Intermittent Failures Robot]

21 failures in 4690 pushes (0.004 failures/push) were associated with this bug in the last 7 days.

This is the #43 most frequent failure this week.

Repository breakdown:

• mozilla-central: 19
• mozilla-beta: 1
• try: 1

Platform breakdown:

• windows10-64-ccov: 19
• macosx1014-64-shippable: 1
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-02-10&endday=2020-02-16&tree=all

Comment 88

[Marco Castelluccio [:marco]]

Yep, they sound reasonable given that it is a coverage build (I would have expected even slower than that!).

Comment 89

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/df278a39cb41
disable create_alwaysMatch.py and timeouts.py in win ccov r=whimboo

Comment 90

[Razvan Maries]

https://hg.mozilla.org/mozilla-central/rev/df278a39cb41

Comment 91

[Intermittent Failures Robot]

7 failures in 4045 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 3
• mozilla-beta: 3
• autoland: 1

Platform breakdown:

• windows10-64-ccov: 3
• windows7-32-devedition: 1
• windows10-64-devedition: 1
• windows10-64-shippable-qr: 1
• linux64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-02-17&endday=2020-02-23&tree=all

Comment 92

[Intermittent Failures Robot]

3 failures in 5233 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-beta: 2
• autoland: 1

Platform breakdown:

• windows7-32-shippable: 3

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-02-24&endday=2020-03-01&tree=all

Comment 93

[Intermittent Failures Robot]

28 failures in 5483 pushes (0.005 failures/push) were associated with this bug in the last 7 days.

This is the #48 most frequent failure this week.

Repository breakdown:

• mozilla-beta: 1
• try: 26
• autoland: 1

Platform breakdown:

• macosx1014-64-devedition: 1
• linux64: 2
• linux64-qr: 1
• windows10-64-qr: 1
• windows10-64-shippable: 7
• windows7-32-shippable: 6
• windows10-64-shippable-qr: 3
• windows7-32-devedition: 4
• windows10-64-devedition: 2
• macosx1014-64-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-03-02&endday=2020-03-08&tree=all

Comment 94

[Intermittent Failures Robot]

17 failures in 976 pushes (0.017 failures/push) were associated with this bug yesterday.

Repository breakdown:

• mozilla-beta: 8
• try: 9

Platform breakdown:

• windows10-64-shippable-qr: 1
• windows10-64-shippable: 6
• windows7-32-shippable: 6
• windows7-32-devedition: 2
• windows7-32: 1
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-03-11&endday=2020-03-11&tree=all

Comment 95

[Intermittent Failures Robot]

25 failures in 989 pushes (0.025 failures/push) were associated with this bug yesterday.

Repository breakdown:

• try: 7
• mozilla-beta: 17
• mozilla-release: 1

Platform breakdown:

• windows10-64-shippable: 8
• linux64-qr: 3
• macosx1014-64-shippable: 1
• macosx1014-64-devedition: 1
• windows10-64-shippable-qr: 5
• windows7-32: 2
• windows7-32-devedition: 3
• windows7-32-shippable: 1
• windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-03-12&endday=2020-03-12&tree=all

Comment 96

[Andreea Pavel [:apavel]]

Recent failures here are bug 1507121.

Comment 97

[Intermittent Failures Robot]

71 failures in 5336 pushes (0.013 failures/push) were associated with this bug in the last 7 days.

This is the #21 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:

• mozilla-beta: 39
• try: 31
• mozilla-release: 1

Platform breakdown:

• windows7-32: 6
• windows10-64-devedition: 11
• macosx1014-64-shippable: 2
• windows10-64-shippable: 21
• windows10-64-shippable-qr: 7
• windows7-32-shippable: 10
• windows7-32-devedition: 7
• linux64-qr: 4
• macosx1014-64-devedition: 2
• linux1804-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-03-09&endday=2020-03-15&tree=all

Comment 98

[Marco Castelluccio [:marco]]

Same for the last Intermittent Failures Robot comment, those failures are not related to this bug.

Comment 99

[Alexandru Michis [:malexandru]]

Marco,
I've reclassified each failure against it's specific Bug.

We should no use this to classify failures anymore, since ccov builds don't run anymore.

Comment 100

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Just clarifying here. We are still running Wd jobs for ccov builds:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=ccov%2Cwd

But a lot of them are still disabled.

Comment 101

[Intermittent Failures Robot]

6 failures in 4955 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 6

Platform breakdown:

• linux1804-64-qr: 1
• macosx1014-64-devedition: 2
• windows10-64-shippable: 1
• windows10-64-devedition: 1
• windows7-32-shippable: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-03-16&endday=2020-03-22&tree=all

Comment 102

[Intermittent Failures Robot]

1 failures in 4130 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• mozilla-central: 1

Platform breakdown:

• windows10-64-ccov: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1495002&startday=2020-10-05&endday=2020-10-11&tree=all

Comment 103

[Intermittent Failures Robot]

1 failures in 3810 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform breakdown:

• windows10-64-qr: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1495002&startday=2021-01-11&endday=2021-01-17&tree=all

Comment 104

[Intermittent Failures Robot]

3 failures in 4622 pushes (0.001 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1
• mozilla-central: 2

Platform and build breakdown:

• linux1804-64-qr: 2
  • opt: 1
  • debug: 1
• linux1804-64-ccov: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1495002&startday=2021-03-08&endday=2021-03-14&tree=all

Comment 105

[Intermittent Failures Robot]

1 failures in 4257 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• autoland: 1

Platform and build breakdown:

• linux1804-64-qr: 1
  • opt: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures/bugdetails?bug=1495002&startday=2021-03-15&endday=2021-03-21&tree=all

Comment 106

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Bug 1418778 re-enabled all the tests.

Comment 107

[BMO Automation]

Moving bug to Remote Protocol::Marionette component per bug 1815831.