521225 - Stop syncing for random failing Places tests, and make sheriffs happier

Status: RESOLVED FIXED

Description

[Marco Bonardo [::mak]]

We can disable the Flush component for random failing tests, that way tests will stop failing, but still test code paths. since we know the reason of the failures but we don't have clear ETA on a proper fix (would require moving most of Places to async stuff), this should allow us to have greener trees.

I'm trying to centralize it in head files so it should be easy to update or remove the workaround.

Comment 1

[Marco Bonardo [::mak]]

Created attachment 405266 [details] [diff] [review]
patch v1.0

This should "fix" about 11 random failures

Comment 2

[Shawn Wilsher :sdwilsh]

Of course, these failing tests are real bugs in our code.  I'm not sure we want to make the tests not test real world behavior.

Comment 3

[Marco Bonardo [::mak]]

once we have identified the cause and we know the fix, and we know we can't fix that in less than 1 year, why should we continue to blame the trees with useless oranges?
this fix is not hiding anything, it's just stopping notifying about a pretty well known bug, for which bugs are and will remain open.
And i think a good part of the bug involves virtual machines, i'm saying that because i see different number of hits for failures in real world again virtual machines,  vms are probably handling threads differently.

Plus the way this is implemented allow us to disable the full thing with really minor changes to try any patch we think can fix the underlying issue.

Comment 4

[Marco Bonardo [::mak]]

the "alternative" as you know, is to disable tests, and that's not better in any way.

Comment 5

[Shawn Wilsher :sdwilsh]

Comment on attachment 405266 [details] [diff] [review]
patch v1.0

Include the bug number where we are tracking the underlying issue in the comment please.

r=sdwilsh

Comment 6

[Marco Bonardo [::mak]]

ugh i was sure to have done that :\

Comment 7

[Marco Bonardo [::mak]]

Created attachment 405656 [details] [diff] [review]
patch v1.1

Comment 8

[Marco Bonardo [::mak]]

http://hg.mozilla.org/mozilla-central/rev/239c5018acec

let's see how this behaves.

Comment 9

[Marco Bonardo [::mak]]

one name was wrong and the test was still reporting failures, fixed.
http://hg.mozilla.org/mozilla-central/rev/0ccc8bcf05f0

looks like random oranges for these tests have stopped. still evaluating and waiting reports though.

Comment 10

[Marco Bonardo [::mak]]

more tests
http://hg.mozilla.org/mozilla-central/rev/12b704972248