1456441 - Intermittent devtools/client/debugger/new/test/mochitest/browser_dbg-pause-exceptions.js | Expected pause line on line 2, it is on 2 -

Status: RESOLVED FIXED

(DevTools :: Debugger, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: ebalazs [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=175265149&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/SDvYGuQESJyU1XbL4jO4Aw/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Intermittent Failures Robot]

16 failures in 168 pushes (0.095 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 7
* mozilla-inbound: 6
* try: 1
* mozilla-central: 1
* mozilla-beta: 1

Platform breakdown:
* windows7-32: 14
* windows7-32-devedition: 1
* windows10-64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-04-26&endday=2018-04-26&tree=trunk

Comment 2

[Intermittent Failures Robot]

19 failures in 127 pushes (0.15 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 12
* mozilla-inbound: 4
* mozilla-beta: 2
* try: 1

Platform breakdown:
* windows7-32: 16
* windows10-64-devedition: 2
* windows7-32-nightly: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-04-27&endday=2018-04-27&tree=trunk

Comment 3

[Tiberius Oros[:tiberius_oros]]

Started doing retriggers for this failure: 
 'devtools/client/debugger/new/test/mochitest/browser_dbg-pause-exceptions.js | Expected pause line on line 2, it is on 2'

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=opt-mochitest-devtools-chrome-e10s&tochange=5579f92920c321f2011db2dbe4118bb3110a46fc&fromchange=9938026038b1d8e5884a61b8ab77ee9aff478ae1&selectedJob=175265149

on the following pushes: 
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=opt-mochitest-devtools-chrome-e10s&tochange=5579f92920c321f2011db2dbe4118bb3110a46fc&fromchange=9938026038b1d8e5884a61b8ab77ee9aff478ae1

and I believe the failure originated from this push: 
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=3ad2e050c66a73c748761d451d2c7a5339501773&filter-searchStr=opt-mochitest-devtools-chrome-e10s

as there are no similar failures on the retriggered jobs.

:jmaher could you please have a look, when you got the time?

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

thanks for doing this Tiberius.  There are a lot of other failures in dt4 which makes this hard to see the failure pattern.  I did some retriggers (30+ on a few revisions) to see if this yields more failures:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=dt4%20win%20pgo&tochange=3ad2e050c66a73c748761d451d2c7a5339501773&fromchange=6bb7337970f19723f9ba34a682b4682c53863e5a

With only 1 or 2 failures in 30 retriggers, it is hard to see a clear pattern.  I would say when those results come back if we do not have at least 4 failures it then becomes difficult to show a clear failure rate.

Comment 5

[Noemi Erli[:noemi_erli]]

After the retriggers were done we noticed that from those three pushes, only the first and second had a single failure on that particular file, the third one had failures but none of them were on the file that we were interested.
Is there a next step in this case?

Comment 6

[Intermittent Failures Robot]

57 failures in 780 pushes (0.073 failures/push) were associated with this bug in the last 7 days. 

This is the #20 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* autoland: 26
* mozilla-inbound: 20
* mozilla-central: 5
* mozilla-beta: 4
* try: 2

Platform breakdown:
* windows7-32: 49
* windows7-32-nightly: 3
* windows10-64-devedition: 3
* windows7-32-devedition: 1
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-04-23&endday=2018-04-29&tree=trunk

Comment 7

[Joel Maher ( :jmaher ) (UTC -8)]

Thanks for working on this Noemi- I am confused why we get so many failures on the trees, yet cannot reproduce this.  I think finding a root cause with retriggers is not straightforward in this case.

Comment 8

[Intermittent Failures Robot]

18 failures in 102 pushes (0.176 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* mozilla-inbound: 10
* autoland: 5
* try: 2
* mozilla-beta: 1

Platform breakdown:
* windows7-32: 16
* windows10-64-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-05-03&endday=2018-05-03&tree=trunk

Comment 9

[Intermittent Failures Robot]

16 failures in 125 pushes (0.128 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* try: 5
* autoland: 5
* mozilla-beta: 4
* mozilla-inbound: 1
* mozilla-central: 1

Platform breakdown:
* windows7-32: 8
* windows7-32-devedition: 3
* windows10-64-devedition: 3
* windows7-32-nightly: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-05-04&endday=2018-05-04&tree=trunk

Comment 10

[Intermittent Failures Robot]

78 failures in 551 pushes (0.142 failures/push) were associated with this bug in the last 7 days. 

This is the #15 most frequent failure this week.   

** This test has failed more than 150 times in the last 21 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* try: 26
* autoland: 21
* mozilla-inbound: 18
* mozilla-beta: 8
* mozilla-central: 5

Platform breakdown:
* windows7-32: 56
* windows10-64-devedition: 9
* windows7-32-nightly: 6
* windows7-32-devedition: 6
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-04-30&endday=2018-05-06&tree=trunk

Comment 11

[Cosmin Sabou [:CosminS]]

Attachment: Disable on Win 7 pgo

Made a patch for this test to be disabled on Win 7 pgo due to the high failure rate. Please review. Thanks.

Comment 12

[Cosmin Sabou [:CosminS]]

Attachment: Disable on Win 7 pgo

Updated the patch with the os version for windows 7.

Comment 13

[Narcis Beleuzu [:NarcisB]]

Comment on attachment 8973615 [details] [diff] [review]
Disable on Win 7 pgo

Review of attachment 8973615 [details] [diff] [review]:
-----------------------------------------------------------------

I would add os_version == "6.1"
skip-if = os = !debug && ("win" && os_version == "6.1") # Bug 1456441

Comment 14

[Cosmin Sabou [:CosminS]]

Attachment: Disable on Win 7 pgo

Corrected the syntax.

Comment 15

[Narcis Beleuzu [:NarcisB]]

Sorry, wanted to say: skip-if = !debug && (os == "win" && os_version == "6.1")

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/efd94fbb9f42
Disable devtools/client/debugger/new/test/mochitest/browser_dbg-pause-exceptions.js on Win 7 pgo. r=nbeleuzu,cbrindusan

Comment 17

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/efd94fbb9f42

Comment 18

[Ryan VanderMeulen (PTO, back 6-April)]

https://hg.mozilla.org/releases/mozilla-beta/rev/eb9cb027960f

Comment 19

[Eliza Balazs [:ebalazs_]]

This still occur on try https://treeherder.mozilla.org/logviewer.html#?job_id=177692815&repo=try&lineNumber=4982
https://treeherder.mozilla.org/#/jobs?repo=try&revision=6c18c182023072c4c18ab0d76cec2a95fd5d7d0b&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable&selectedJob=177692815

Comment 20

[Cosmin Sabou [:CosminS]]

This is not fixed, I just made a patch to disable the test because of the high failure rate. It was disabled only on win 7 pgo.

Comment 21

[Jason Laster [:jlast]]

Thanks i believe we have a fix for 62 nightly. It was difficult to hunt down, but i believe was effecting other tests a bit as well.

Comment 22

[Intermittent Failures Robot]

12 failures in 660 pushes (0.018 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 6
* try: 5
* autoland: 1

Platform breakdown:
* windows10-64-devedition: 8
* windows7-32-nightly: 1
* windows7-32: 1
* windows10-64: 1
* linux64-devedition: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-05-07&endday=2018-05-13&tree=trunk

Comment 23

[Intermittent Failures Robot]

7 failures in 639 pushes (0.011 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 6
* try: 1

Platform breakdown:
* windows10-64-devedition: 7

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-05-14&endday=2018-05-20&tree=trunk

Comment 24

[Intermittent Failures Robot]

2 failures in 632 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-beta: 2

Platform breakdown:
* windows10-64-devedition: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1456441&startday=2018-05-21&endday=2018-05-27&tree=trunk