Intermittent test_safe_browsing_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | TimeoutException: Timed out after 30.1 seconds with message: Safe Browsing File: ^goog-badbinurl-shavar.pset$ not found!

RESOLVED FIXED in mozilla51

Status

RESOLVED FIXED
2 years ago
a year ago

People

(Reporter: intermittent-bug-filer, Unassigned)

Tracking

({intermittent-failure})

Version 3
mozilla51
intermittent-failure
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Downloaded safebrowsing files: ['test-unwanted-simple.pset', 'test-block-simple.pset', 'test-malware-simple.pset', 'test-trackwhite-simple.pset', 'test-track-simple.sbstore', 'test-phish-simple.pset', 'test-phish-simple.sbstore', 'test-trackwhite-simple.sbstore', 'test-malware-simple.sbstore', 'test-track-simple.pset', 'test-unwanted-simple.sbstore', 'test-block-simple.sbstore']
Depends on: 1302958
No longer depends on: 1302958
Depends on: 1304983
Francois, fyi we are failing a lot in this test on Aurora those days mainly on OS X and Windows, but not Linux. It happens across locales.

Maybe someone should investigate?
Flags: needinfo?(francois)
(In reply to Henrik Skupin (:whimboo) from comment #2)
> Francois, fyi we are failing a lot in this test on Aurora those days mainly
> on OS X and Windows, but not Linux. It happens across locales.
> 
> Maybe someone should investigate?

Is it just slow? Depending on the kind of VM and network that these tests run on, it's possible we need to give the tests a minute or two to download the lists.
Flags: needinfo?(francois)
Matt, would you be able to confirm that Safe Browsing is working in Aurora on OSX and Windows, just in case?

I'm thinking:

1. creating a new profile
2. checking that the list files have been downloaded to the cache directory (~/Library/Caches/Firefox/Profiles/XXXX/safebrowsing/ and C:\Users\XXXX\AppData\Local\mozilla\firefox\profiles\XXXX\safebrowsing\) after a few minutes
3. checking the Webpage Warnings links on https://testsafebrowsing.appspot.com/ after a few hours
Flags: needinfo?(mwobensmith)
21 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-aurora: 21

Platform breakdown:
* windows7-64: 4
* osx-10-9: 4
* windows8-64: 3
* osx-10-11: 3
* windowsxp: 2
* windows8-32: 2
* windows7-32: 1
* osx-10-10: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1304781&startday=2016-10-12&endday=2016-10-12&tree=all
(In reply to François Marier [:francois] from comment #3)
> (In reply to Henrik Skupin (:whimboo) from comment #2)
> Is it just slow? Depending on the kind of VM and network that these tests
> run on, it's possible we need to give the tests a minute or two to download
> the lists.

It's hard to tell without the changes from bug 1304983. Should we maybe backport this addition to aurora?
(In reply to Henrik Skupin (:whimboo) from comment #6)
> It's hard to tell without the changes from bug 1304983. Should we maybe
> backport this addition to aurora?

Right, I had forgotten that the test on Aurora was different from the one in Nightly. I think it makes sense to uplift given that this test has caused problems in the past.
Ok, I requested the uplift of the changes. It will also mean we won't hit a failure signature like this one anymore once landed. Once we see a similar bug I will duplicate it accordingly. Maybe its related to bug 1309764.
See Also: → bug 1309764
Francois, on mozilla-aurora the first failures clearly started between those two nightly builds:

https://hg.mozilla.org/releases/mozilla-aurora/pushloghtml?fromchange=41593448b1ae&tochange=fb564edbd98d

Not sure what this "Automated HSTS preload list update" is, but maybe it is related? I will try to retrigger some tests from before those changes to see if those tests are still green. If not we might see a remote change by Google?
(In reply to Henrik Skupin (:whimboo) from comment #9)
> Not sure what this "Automated HSTS preload list update" is, but maybe it is
> related? I will try to retrigger some tests from before those changes to see
> if those tests are still green.

Interesting. The HSTS preload list is not delivered through Safe Browsing. I would guess it's delivered by Kinto.

Maybe it affects the timing of other downloads that happen at startup time? Would disabling network.stricttransportsecurity.preloadlist actually change anything?

> If not we might see a remote change by Google?

If that were the case then we'd see manual tests also fail. Hopefully Matt can confirm that this is not happening.
(In reply to Henrik Skupin (:whimboo) from comment #9)
> https://hg.mozilla.org/releases/mozilla-aurora/
> pushloghtml?fromchange=41593448b1ae&tochange=fb564edbd98d
> 
> Not sure what this "Automated HSTS preload list update" is, but maybe it is
> related?

Oh, I was wrong. This isn't about letting kinto ship HPKP/HSTS lists to Firefox, but rather about doing a sync of the Google list.

Nothing in that pushlog seems related to the timeouts we're seeing.
27 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-aurora: 27

Platform breakdown:
* windows8-64: 6
* windows7-64: 6
* windowsxp: 3
* linux32: 3
* windows8-32: 2
* windows7-32: 2
* osx-10-11: 2
* linux64: 2
* osx-10-9: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1304781&startday=2016-10-14&endday=2016-10-14&tree=all
63 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-aurora: 63

Platform breakdown:
* windows7-64: 13
* windows8-64: 12
* osx-10-9: 7
* osx-10-11: 7
* windowsxp: 6
* linux32: 5
* windows8-32: 4
* windows7-32: 3
* osx-10-10: 3
* linux64: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1304781&startday=2016-10-10&endday=2016-10-16&tree=all
Using steps from comment 4, I see no issues with today's Aurora 51.0a2, on Windows 10 and Mac OSX. As always, the updates take several hours, but eventually the sites are blocked.
Flags: needinfo?(mwobensmith)
Status: NEW → RESOLVED
Last Resolved: a year ago
Keywords: bulk-close-intermittents
Resolution: --- → INCOMPLETE
Fixed by bug 1304983 for Firefox 51.0.
Resolution: INCOMPLETE → FIXED
Target Milestone: --- → mozilla51
Keywords: bulk-close-intermittents
You need to log in before you can comment on or make changes to this bug.