Closed Bug 950025 Opened 7 years ago Closed 6 years ago

Test failure "Blocklist has been updated." in add-on blocklist tests

Categories

(Mozilla QA Graveyard :: Mozmill Tests, defect, P2)

defect

Tracking

(firefox27 wontfix, firefox28 wontfix, firefox29 wontfix, firefox30 wontfix, firefox31 unaffected, firefox32 unaffected, firefox33 fixed)

RESOLVED FIXED
Tracking Status
firefox27 --- wontfix
firefox28 --- wontfix
firefox29 --- wontfix
firefox30 --- wontfix
firefox31 --- unaffected
firefox32 --- unaffected
firefox33 --- fixed

People

(Reporter: AndreeaMatei, Assigned: andrei)

References

()

Details

(Keywords: regression, Whiteboard: [mozmill-test-failure])

Attachments

(2 files)

Failed with Windows 8, default, de locale, also on test4.js:
http://mozmill-daily.blargon7.com/#/functional/report/850f11101678d9fbefeb95e0bd4debb1
OS: Linux → All
Firefox 28.0 (28.0, pl, 20140218122424) - Linux Ubuntu 12.04 (x86_64):
http://mozmill-release.blargon7.com/#/functional/report/6062fdddb60e88657bc78ec1f4185f11
This might be a network glitch we are facing here. I have seen the same issue locally while my MBP was not able to connect to my local network. Means no network connection is available. So it seems that resetting the blacklist somehow really needs a working network connection.

Something we should check here is to also reset PREF_BLOCKLIST_PINGCOUNTTOTAL, PREF_BLOCKLIST_PINGCOUNTVERSION, and PREF_BLOCKLIST_LASTUPDATETIME. Given that the hard block test passes but the soft block fails, which directly comes afterward. 

http://dxr.mozilla.org/mozilla-central/source/toolkit/mozapps/extensions/nsBlocklistService.js#454

Otherwise it could be related to httpd.js and not serving the right page. Someone might check this first if an JS error is visible in the browser console.
Failed over the weekend once on windows 8, with aurora it locale:
http://mozmill-daily.blargon7.com/#/functional/report/e248c845ffb052c61b3635551bc1a6a2
Priority: P3 → P4
Failed again once with Aurora fr, on Windows 8 32 (mm-win-8-32-4).
http://mozmill-daily.blargon7.com/#/functional/report/2686ef5c08d17b800fd2f557bf019bca
Still failing:
http://mozmill-daily.blargon7.com/#/functional/report/7ab760e27012969ae02e3b9e41408335

Current failure rate (roughly average) about 1 / week (for the last 3 months).
The failures themselves are not evenly distributed. They are more or less clustered.
Depends on: 987612
Summary: Test failure "Blocklist has been updated." in testAddons_installUninstallSoftBlocklistedExtension/test4.js → Test failure "Blocklist has been updated." in add-on blocklist tests
Duplicate of this bug: 1008830
Failed 4 times with Aurora de on Mac OSX's nodes.
We may want to hurry up with fixing bug 987612 ?
Attached patch skip.patchSplinter Review
This started to fail constantly in /restartTests/testAddons_installUninstallSoftBlocklistedExtension/test4.js

Skip patch provided.
We will investigate this sooner.
Attachment #8455210 - Flags: review?(andrei.eftimie)
We may want to also check bug 987612 for this.
Running just the test it doesn't reproduce. Only testruns so it may depend on previous tests.
Will continue the investigation in 1h.
Comment on attachment 8455210 [details] [diff] [review]
skip.patch

Review of attachment 8455210 [details] [diff] [review]:
-----------------------------------------------------------------

Disabled:
https://hg.mozilla.org/qa/mozmill-tests/rev/a59e61505aef (default)
Attachment #8455210 - Flags: review?(andrei.eftimie)
Attachment #8455210 - Flags: review+
Attachment #8455210 - Flags: checkin+
Going to investigate this and get a regression range.
Assignee: nobody → andrei.eftimie
Status: NEW → ASSIGNED
This is profile-related. I can reliably reproduce this by running both:
> testAddons_installUninstallHardBlocklistedExtension
> testAddons_installUninstallSoftBlocklistedExtension
one after the other with a --profile set.

Does not fail if I don't specify a custom profile.
Attached file testcase.zip
Testcase attached.

Unzip, then run > mozmill -t testcase -b %firefox% --profile=%profile%
The 3rd test will fail.

Seems we need to reset the profile twice whilest pinging the blacklist service inbetween restarts. It might that it can further be reduced, but I wasn't able to properly reproduce it with only 2 restart instances. With 3 it fails every time.

I'll be reducing the regression range now.
I'll also disect the profile to see if I find anything interesting.
mc pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=cb75d6cfb004&tochange=e1a037c085d1

working with tinderbox builds to find the actual changeset
Keywords: regression
We have seen this problem already with Firefox 27 as what the status flags say. So how can this be a regression in Firefox 33? Maybe you are seeing something different?
Inbound pushlog:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=19663c22e7eb&tochange=3a4d57c7ffdf

As a regressor we have bug 1035942 or bug 1035009

These are mostly certificate-related changes. Not sure how this ties in with the failure we see.
We fail to reconnect to a previously made secure connection?!

==

Brian, do you know how the changes mentioned above could affect Firefox's Blocklist service?
Basically this fails when we try to get an updated blocklist from Mozilla's servers.

This is the code that fails (only fails when some profile conditions are met, basically have to call this 3 times and clean the profile between the calls):
>  var done = false;
>  Services.obs.addObserver(() => { done = true }, "blocklist-updated", false);
>  var blocklistService = Cc["@mozilla.org/extensions/blocklist;1"]
>                         .getService(Ci.nsIBlocklistService);
>  blocklistService.QueryInterface(Ci.nsITimerCallback).notify(null);
>  expect.waitFor(() => done, "Blocklist has been updated.")

I'm still working on figuring out the details (profile and network wise).
Flags: needinfo?(brian)
(In reply to Henrik Skupin (:whimboo) [away 07/19 - 08/01] from comment #24)
> We have seen this problem already with Firefox 27 as what the status flags
> say. So how can this be a regression in Firefox 33? Maybe you are seeing
> something different?

Very possible that this is a different issue with the same visible outcome than the original problem. We have a reliable testcase with a regressor as mentioned in comment 25 (from July 11th on mc).

The original mentioned issue was intermittent with a low-occurrence rate (could have also been network related).

I propose to fix the current issue then monitor if we still have intermittent failures...
Please see bug 1038098. In particular, please check whether the fix for bug 1038098 fixes this bug.
Depends on: 1038098
Flags: needinfo?(brian)
Thanks Brian, indeed, with the latest Nighlty which includes the fix from bug 1038098 I can no longer reproduce the failure.

Backed out skip:
https://hg.mozilla.org/qa/mozmill-tests/rev/c976bd157a00 (default)

I'll tentatively mark this as 'fixed' on 33, please do update if the blocklist fails again (I assume it will in the near future). If that happens we should focus on bug 987612 to get a fix.
This hasn't failed since, reducing priority at least.
Priority: P1 → P2
We haven't recorded a single failure since Brian's patch landed.
Last failures were recorded on the 13th July.
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.