Intermittent TEST-UNEXPECTED-PASS | dom/base/test/browser_use_counters.js | page counts for PROPERTY_FILLOPACITY after are correct - Got 1, expected 1

RESOLVED FIXED in Firefox 52

Status

()

Core
DOM
RESOLVED FIXED
2 years ago
a year ago

People

(Reporter: Treeherder Bug Filer, Assigned: froydnj)

Tracking

({intermittent-failure})

unspecified
mozilla53
intermittent-failure
Points:
---
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox51 unaffected, firefox52 fixed, firefox53 fixed)

Details

Attachments

(1 attachment)

Comment 1

a year ago
11 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 7
* autoland: 3
* mozilla-central: 1

Platform breakdown:
* linux64: 9
* linux32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-07&endday=2016-11-13&tree=all

Comment 2

a year ago
24 failures in 124 pushes (0.194 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 12
* mozilla-inbound: 10
* mozilla-central: 2

Platform breakdown:
* linux64: 18
* linux32: 5
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-14&endday=2016-11-14&tree=all

Comment 3

a year ago
19 failures in 144 pushes (0.132 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* mozilla-inbound: 10
* autoland: 8
* mozilla-central: 1

Platform breakdown:
* linux64: 12
* linux32: 5
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-15&endday=2016-11-15&tree=all

Comment 4

a year ago
19 failures in 119 pushes (0.16 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 18
* mozilla-central: 1

Platform breakdown:
* linux64: 14
* linux32: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-16&endday=2016-11-16&tree=all

Comment 5

a year ago
78 failures in 715 pushes (0.109 failures/push) were associated with this bug in the last 7 days. 

This is the #30 most frequent failure this week. 

** This failure happened more than 50 times this week! Resolving this bug is a high priority. **

Repository breakdown:
* autoland: 41
* mozilla-inbound: 22
* mozilla-central: 8
* try: 4
* mozilla-aurora: 2
* graphics: 1

Platform breakdown:
* linux64: 53
* linux32: 18
* osx-10-10: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-14&endday=2016-11-20&tree=all

Comment 6

a year ago
27 failures in 124 pushes (0.218 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* mozilla-inbound: 21
* autoland: 4
* mozilla-central: 2

Platform breakdown:
* linux64: 24
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-23&endday=2016-11-23&tree=all

Comment 7

a year ago
16 failures in 114 pushes (0.14 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 12
* mozilla-inbound: 3
* mozilla-central: 1

Platform breakdown:
* linux64: 13
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-24&endday=2016-11-24&tree=all

Comment 8

a year ago
16 failures in 72 pushes (0.222 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 8
* mozilla-inbound: 4
* mozilla-central: 2
* try: 1
* mozilla-aurora: 1

Platform breakdown:
* linux64: 11
* osx-10-10: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-25&endday=2016-11-25&tree=all

Comment 9

a year ago
90 failures in 623 pushes (0.144 failures/push) were associated with this bug in the last 7 days. 

This is the #10 most frequent failure this week. 

** This failure happened more than 50 times this week! Resolving this bug is a high priority. **

Repository breakdown:
* mozilla-inbound: 48
* autoland: 33
* mozilla-central: 7
* try: 1
* mozilla-aurora: 1

Platform breakdown:
* linux64: 72
* linux32: 12
* osx-10-10: 6

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-21&endday=2016-11-27&tree=all
Nathan, this is our #3 intermittent orange over the last 3 days, I see the error as [1]:
TEST-UNEXPECTED-PASS | dom/base/test/browser_use_counters.js | page counts for PROPERTY_FILLOPACITY after are correct - Got 1, expected 1


this is very frequent on linux64 debug and asan. Can you take a look at this and figure out what is going on?


[1] https://treeherder.mozilla.org/logviewer.html#?repo=mozilla-inbound&job_id=39906828#L5960
Flags: needinfo?(nfroyd)
(Assignee)

Comment 11

a year ago
Created attachment 8815066 [details] [diff] [review]
avoid using todo_is in browser_use_counters.js

Feel free to redirect the review to someone else, but as you've reviewed the
previous intermittent fix in this test...
Attachment #8815066 - Flags: review?(dholbert)
(Assignee)

Updated

a year ago
Assignee: nobody → nfroyd
Flags: needinfo?(nfroyd)
Do you know *why* we'd be unexpectedly (and occasionally) passing this particular check?

Presumably we had this "todo_is" here for a reason.  If its assumptions are being violated, that seems maybe-worrisome.  (Though maybe the "todo_is" is here for a bad reason, and really we just have no confidence about the value [correct or incorrect] when xfail=true?)
Flags: needinfo?(nfroyd)
31 failures in 110 pushes (0.282 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 14
* mozilla-inbound: 13
* mozilla-central: 3
* try: 1

Platform breakdown:
* linux64: 25
* linux32: 6

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-28&endday=2016-11-28&tree=all
(Assignee)

Comment 14

a year ago
(In reply to Daniel Holbert [:dholbert] from comment #12)
> Do you know *why* we'd be unexpectedly (and occasionally) passing this
> particular check?
> 
> Presumably we had this "todo_is" here for a reason.  If its assumptions are
> being violated, that seems maybe-worrisome.  (Though maybe the "todo_is" is
> here for a bad reason, and really we just have no confidence about the value
> [correct or incorrect] when xfail=true?)

check_use_counter_direct is used to test that SVGs loaded directly update their use counters.  For the test where we pass xfail=true, we have an SVG document (call this A) with a fill pattern defined in a different SVG document (call this B).  B is the document that touches the CSS property we're actually interested in for the test in question, and the histograms we're interested in will only get touched when B gets destroyed.  We'd normally work around this race by eagerly updating the use counters for B from the test, but we don't have direct access to B, only to A--whose use counters we *do* eagerly update.  When B updates the interesting histograms is then a question of when B gets destroyed.

So these few UNEXPECTED-PASS results are happening because the race is working out not in our favor on these slower test machines (debug and asan).  I don't think we have direct access to B from A for eagerly updating the use counters, which would be a better solution if it were possible.
Flags: needinfo?(nfroyd)
Thanks for explaining that!

Perhaps we should just change our C++ code to make forced-use-counter-flushes for A implicitly also flush use counters for B? (Under the hood, the actual nsDocument for "A" here *does* have access to the nsDocument for "B". See e.g. nsDocument::FlushExternalResources and nsDocument::OnPageShow, each of which call "EnumerateExternalResources" to notifify all of the external-resource-docs like "B".)

That doesn't seem too hard, and it'd allow the test to keep functioning as-expected, rather than skipping a bunch of testing in a particular case...
37 failures in 170 pushes (0.218 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 20
* mozilla-inbound: 14
* mozilla-central: 2
* try: 1

Platform breakdown:
* linux64: 29
* linux32: 6
* osx-10-10: 1
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-29&endday=2016-11-29&tree=all
ni for comment 15. (Let me know if you'd like me to take a crack at it, too - I didn't necessarily mean that you'd have to be the person who handles what I'm suggesting in comment 15. Though I think it should be pretty straightforward.)
Flags: needinfo?(nfroyd)
30 failures in 131 pushes (0.229 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 15
* mozilla-inbound: 12
* mozilla-central: 2
* try: 1

Platform breakdown:
* linux64: 23
* linux32: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-30&endday=2016-11-30&tree=all
20 failures in 133 pushes (0.15 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 11
* mozilla-inbound: 6
* try: 3

Platform breakdown:
* linux64: 17
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-12-01&endday=2016-12-01&tree=all
17 failures in 87 pushes (0.195 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* mozilla-inbound: 8
* autoland: 7
* try: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 12
* linux32: 4
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-12-02&endday=2016-12-02&tree=all
184 failures in 694 pushes (0.265 failures/push) were associated with this bug in the last 7 days. 

This is the #3 most frequent failure this week. 

** This failure happened more than 50 times this week! Resolving this bug is a high priority. **

Repository breakdown:
* autoland: 96
* mozilla-inbound: 63
* mozilla-central: 17
* try: 8

Platform breakdown:
* linux64: 142
* linux32: 38
* osx-10-10: 3
* android-4-3-armv7-api15: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1317046&startday=2016-11-28&endday=2016-12-04&tree=all
(In reply to Daniel Holbert [:dholbert] from comment #12)
> occasionally
(In reply to Nathan Froyd [:froydnj] from comment #14)
> few

Currently our number 1 active failure, and based on some retriggering it fails 5-10% of the time on the more affected platforms, which is more than enough to justify disabling the entire test. I'd recommend taking a patch to disable a tiny part of the test, immediately, and only then worry about how to make it better.
Flags: needinfo?(dholbert)
Comment on attachment 8815066 [details] [diff] [review]
avoid using todo_is in browser_use_counters.js

Review of attachment 8815066 [details] [diff] [review]:
-----------------------------------------------------------------

Fair enough. Could you file a followup on comment 15, though, to fix this correctly?

In the meantime, I suppose this seems fine.
Attachment #8815066 - Flags: review?(dholbert) → review+
(In reply to Daniel Holbert [:dholbert] from comment #23)
> Fair enough. Could you file a followup on comment 15, though, to fix this
> correctly?

("you" there is directed at froydnj.)
Flags: needinfo?(dholbert)

Comment 25

a year ago
Pushed by nfroyd@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/590c70d2662a
avoid using todo_is in browser_use_counters.js; r=dholbert
(Assignee)

Comment 26

a year ago
(In reply to Daniel Holbert [:dholbert] from comment #23)
> Fair enough. Could you file a followup on comment 15, though, to fix this
> correctly?

ni? myself to make sure I do this on Tuesday.

> In the meantime, I suppose this seems fine.

Thank you!
Flags: needinfo?(nfroyd)
(Assignee)

Comment 27

a year ago
Try that again. :(
Flags: needinfo?(nfroyd)

Comment 28

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/590c70d2662a
Status: NEW → RESOLVED
Last Resolved: a year ago
status-firefox53: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
(Assignee)

Comment 29

a year ago
(In reply to Daniel Holbert [:dholbert] from comment #23)
> Fair enough. Could you file a followup on comment 15, though, to fix this
> correctly?

Filed bug 1322396.
Flags: needinfo?(nfroyd)
status-firefox51: --- → unaffected
status-firefox52: --- → affected

Comment 30

a year ago
bugherderuplift
https://hg.mozilla.org/releases/mozilla-aurora/rev/cf05a9412d74
status-firefox52: affected → fixed
Flags: in-testsuite+
You need to log in before you can comment on or make changes to this bug.