Intermittent toolkit/content/tests/chrome/test_bug263683.xul | Correctly added a match to the selection type - 0 == 1, | Test timed out

NEW
Assigned to

Status

()

2 years ago
2 months ago

People

(Reporter: intermittent-bug-filer, Assigned: mikedeboer)

Tracking

({intermittent-failure, leave-open})

unspecified
mozilla54
intermittent-failure, leave-open
Points:
---
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox52 fixed, firefox-esr52 fixed, firefox53 fixed, firefox54 fixed)

Details

(Whiteboard: [stockwell disabled])

MozReview Requests

()

Submitter Diff Changes Open Issues Last Updated
Loading...
Error loading review requests:

Attachments

(2 attachments)

31 failures in 749 pushes (0.041 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* mozilla-inbound: 14
* autoland: 11
* mozilla-central: 3
* try: 2
* graphics: 1

Platform breakdown:
* linux64: 15
* linux32: 9
* windows7-32: 4
* osx-10-10: 2
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-01-23&endday=2017-01-29&tree=all
This bug and bug 1333299 seem to have started around January 23 - about 10 days after the test was updated in bug 1316514.

Mike - can you have a look at these?
Flags: needinfo?(mdeboer)
51 failures in 733 pushes (0.07 failures/push) were associated with this bug in the last 7 days. 

This is the #29 most frequent failure this week. 

** This failure happened more than 50 times this week! Resolving this bug is a high priority. **

Repository breakdown:
* mozilla-inbound: 22
* autoland: 20
* try: 6
* mozilla-central: 3

Platform breakdown:
* linux64: 29
* windows7-32: 14
* linux32: 6
* windows8-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-01-30&endday=2017-02-05&tree=all
I would really like to see this bug fixed in the next week or so.  If we don't have time or this goes much into next week without progress, I would advocate for disabling this test until we have more time to work on it.  This is primarily on Linux and Windows (not osx/android) and between this and bug 1333299 this test would be in the top 5 failures in the last couple weeks.
(Assignee)

Comment 5

2 years ago
I'm planning to do just that.
(Assignee)

Updated

2 years ago
Assignee: nobody → mdeboer
Status: NEW → ASSIGNED
(Assignee)

Updated

2 years ago
Flags: needinfo?(mdeboer)
Comment hidden (mozreview-request)
Comment on attachment 8834965 [details]
Bug 1333270 - disable remote browser testing in test_bug263683.xul due to frequent intermittent failures.

https://reviewboard.mozilla.org/r/110700/#review112118

rs=me
Attachment #8834965 - Flags: review?(jaws) → review+

Comment 10

2 years ago
Pushed by mdeboer@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6f53d355032d
fix frequently failing test_bug263683.xul by reordering the sequence of events on a fresh document. r=jaws

Comment 11

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/6f53d355032d
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
status-firefox54: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
status-firefox52: --- → affected
status-firefox53: --- → affected

Comment 12

2 years ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-aurora/rev/3ae4ed8cb4ab
status-firefox53: affected → fixed
Flags: in-testsuite+

Comment 13

2 years ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-beta/rev/8cb9123ea459
status-firefox52: affected → fixed
20 failures in 153 pushes (0.131 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* mozilla-inbound: 9
* autoland: 9
* try: 1
* graphics: 1

Platform breakdown:
* windows7-32: 11
* osx-10-10: 6
* windows8-64: 2
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-09&endday=2017-02-09&tree=all
15 failures in 152 pushes (0.099 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* mozilla-inbound: 9
* autoland: 5
* mozilla-central: 1

Platform breakdown:
* windows7-32: 9
* osx-10-10: 2
* linux64: 2
* windows8-64: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-10&endday=2017-02-10&tree=all
FWIW, this appears to be "fixed" in the sense of "occurs exactly as frequently as it did before, but because the bug is closed treeherder hides it behind a toggle, so it's now a duplicate of bug 1333299 since that's still open so everyone but philor just stars it as that one."
16 failures in 46 pushes (0.348 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* autoland: 9
* mozilla-inbound: 7

Platform breakdown:
* windows7-32: 7
* linux64: 4
* windows8-64: 2
* osx-10-10: 2
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-11&endday=2017-02-11&tree=all
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Summary: Intermittent toolkit/content/tests/chrome/test_bug263683.xul | Correctly added a match to the selection type - 0 == 1 → Intermittent toolkit/content/tests/chrome/test_bug263683.xul | Correctly added a match to the selection type - 0 == 1, | Test timed out
Duplicate of this bug: 1333299
176 failures in 836 pushes (0.211 failures/push) were associated with this bug in the last 7 days. 

This is the #7 most frequent failure this week. 

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:
* mozilla-inbound: 86
* autoland: 72
* mozilla-central: 9
* try: 6
* graphics: 3

Platform breakdown:
* windows7-32: 76
* linux64: 39
* osx-10-10: 25
* linux32: 20
* windows8-64: 16

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-06&endday=2017-02-12&tree=all
Flags: needinfo?(mdeboer)
64 failures in 114 pushes (0.561 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* mozilla-inbound: 31
* autoland: 29
* graphics: 2
* try: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 23
* windows7-32: 19
* osx-10-10: 13
* linux32: 6
* windows8-64: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-13&endday=2017-02-13&tree=all
(Assignee)

Comment 21

2 years ago
I don't like to see this test disabled - I'm working on it right now.
Status: REOPENED → ASSIGNED
Flags: needinfo?(mdeboer)
(Assignee)

Comment 22

2 years ago
There's a problem with the tests' continuation with the remote (e10s) browser element. This is not an e10s problem, but rather a timing-related issue.
There should be an easy way to fix this, upcoming try pushes will tell.
Comment hidden (mozreview-request)
(Assignee)

Comment 25

2 years ago
This patch changed my windows VM from perma-ish-failing to always-passing. I expect the try build to display similar results and will retrigger plenty there.
(Assignee)

Comment 26

2 years ago
Well, win7 debug doesn't comply. Trying something else then.

Comment 27

2 years ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-esr52/rev/8cb9123ea459
status-firefox-esr52: --- → fixed
65 failures in 161 pushes (0.404 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* mozilla-inbound: 32
* autoland: 25
* mozilla-central: 7
* try: 1

Platform breakdown:
* linux64: 24
* windows7-32: 21
* linux32: 9
* osx-10-10: 6
* windows8-64: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-14&endday=2017-02-14&tree=all
37 failures in 143 pushes (0.259 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* mozilla-inbound: 18
* autoland: 17
* try: 2

Platform breakdown:
* windows7-32: 15
* linux64: 12
* osx-10-10: 4
* windows8-64: 3
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-15&endday=2017-02-15&tree=all
Comment hidden (mozreview-request)
(Assignee)

Updated

2 years ago
Attachment #8834965 - Flags: review+ → review?(jaws)
(Assignee)

Comment 32

2 years ago
Making this change in the test made it very stable on OSX, but Win/ Lin _debug_ remained spotty when switching to the remote browser and running the same tests for it. I don't know why, can't reproduce it locally at all, thus consider it not worth the immense trouble.
Additionally, the try runs show that it's also quite hard to trigger on infra, with this fix.

Sound reasonable?
Comment on attachment 8834965 [details]
Bug 1333270 - disable remote browser testing in test_bug263683.xul due to frequent intermittent failures.

https://reviewboard.mozilla.org/r/110700/#review114600
Attachment #8834965 - Flags: review?(jaws) → review+
49 failures in 151 pushes (0.325 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* mozilla-inbound: 24
* autoland: 17
* mozilla-central: 4
* graphics: 3
* try: 1

Platform breakdown:
* windows7-32: 16
* linux64: 16
* osx-10-10: 7
* windows8-64: 6
* linux32: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-16&endday=2017-02-16&tree=all

Comment 35

2 years ago
Pushed by mdeboer@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/d7bf5b3165ad
wait until we find a word before we yield/ await the toggle highlight promise to avoid timing issues when switching to a remote browser element. r=jaws

Comment 36

2 years ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/d7bf5b3165ad
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago2 years ago
Resolution: --- → FIXED
59 failures in 180 pushes (0.328 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* autoland: 32
* mozilla-inbound: 21
* mozilla-central: 3
* graphics: 3

Platform breakdown:
* windows8-64: 24
* osx-10-10: 19
* linux64: 9
* linux32: 6
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-17&endday=2017-02-17&tree=all
310 failures in 833 pushes (0.372 failures/push) were associated with this bug in the last 7 days. 

This is the #8 most frequent failure this week. 

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:
* mozilla-inbound: 140
* autoland: 131
* mozilla-central: 19
* try: 10
* graphics: 10

Platform breakdown:
* linux64: 93
* windows7-32: 85
* osx-10-10: 58
* windows8-64: 44
* linux32: 30

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-13&endday=2017-02-19&tree=all
A dozen failures already today, doesn't seem very fixed.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
15 failures in 141 pushes (0.106 failures/push) were associated with this bug yesterday.  
Repository breakdown:
* autoland: 6
* mozilla-inbound: 5
* try: 2
* mozilla-central: 2

Platform breakdown:
* osx-10-10: 7
* linux64: 6
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-21&endday=2017-02-21&tree=all
it looks like the failures are greatly reduced, but still at a rate of >60/week, primarily on opt/pgo non-e10s.

Mike can you take another look at this?
Flags: needinfo?(mdeboer)
72 failures in 812 pushes (0.089 failures/push) were associated with this bug in the last 7 days. 

This is the #18 most frequent failure this week. 

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:
* autoland: 25
* mozilla-inbound: 24
* mozilla-central: 13
* try: 6
* graphics: 4

Platform breakdown:
* osx-10-10: 36
* linux64: 25
* linux32: 10
* windows8-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-20&endday=2017-02-26&tree=all
Created attachment 8842157 [details] [diff] [review]
temporarily disable this until we can get a fix in place

1.5 weeks since we landed the fix, but no luck in getting more traction here.  Lets disable this until we have more time to hack on this.
Attachment #8842157 - Flags: review?(gbrown)
Attachment #8842157 - Flags: review?(gbrown) → review+
Comment hidden (mozreview-request)
(Assignee)

Updated

2 years ago
Flags: needinfo?(mdeboer)
I would rather use the mozreview attachment than mine- thanks for putting that together Mike.

Comment 46

2 years ago
mozreview-review
Comment on attachment 8834965 [details]
Bug 1333270 - disable remote browser testing in test_bug263683.xul due to frequent intermittent failures.

https://reviewboard.mozilla.org/r/110700/#review117828

Are we going to either leave this bug open or file a followup to get this fixed properly?
Attachment #8834965 - Flags: review?(gijskruitbosch+bugs) → review+
(Assignee)

Comment 47

2 years ago
I think leaving this open would be appropriate.

Updated

2 years ago
Keywords: leave-open
(Assignee)

Updated

2 years ago
Status: REOPENED → NEW

Comment 48

2 years ago
Pushed by mdeboer@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6b64847df216
disable remote browser testing in test_bug263683.xul due to frequent intermittent failures. r=Gijs,jaws
marking as disabled for current state as we have disabled part of the test- leave-open is set on this bug, so we are good.
Whiteboard: [stockwell disabled]
25 failures in 783 pushes (0.032 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 12
* mozilla-inbound: 4
* graphics: 4
* mozilla-central: 3
* try: 1
* oak: 1

Platform breakdown:
* linux64: 11
* osx-10-10: 10
* linux32: 4

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1333270&startday=2017-02-27&endday=2017-03-05&tree=all
1 failures in 660 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-inbound: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1333270&startday=2018-05-07&endday=2018-05-13&tree=trunk
1 failures in 632 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1333270&startday=2018-05-21&endday=2018-05-27&tree=trunk
1 failures in 655 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1333270&startday=2018-05-28&endday=2018-06-03&tree=trunk
1 failures in 669 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1333270&startday=2018-06-04&endday=2018-06-10&tree=trunk
You need to log in before you can comment on or make changes to this bug.