1399311 - Intermittent test_safe_browsing_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: 'goog-badbinurl-proto.pset' not found in []

Status: RESOLVED FIXED

(Toolkit :: Safe Browsing, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: rvandermeulen [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=130506456&repo=mozilla-beta

https://queue.taskcluster.net/v1/task/YmdXFUg4TCivV2S9w_wPUA/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Francois, maybe it's a failure which we already fixed on central?

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Maybe similar to bug 1399311 caused by non responding Google servers? It started all with this changeset but I cannot find any safebrowsing changes in there:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=867b9e3b4f92445085c77cecf485f6a5568faeaf&filter-searchStr=Firefox%20UI&filter-tier=1&filter-tier=2&filter-tier=3

Comment 3

[Thomas Nguyen (:tnguyen)]

Hmm, I think that would be we roll back to V2 in release, but we did not update the test.
It goes to V2 update but test V4 tables
Should be bug 1393954

Comment 4

[Thomas Nguyen (:tnguyen)]

Hmm, I think that would be we did roll back to V2 in beta, but we did not update the test.
It goes to V2 update but tests V4 tables
Should be bug 1393954

Comment 5

[Thomas Nguyen (:tnguyen)]

Oh, it seems the bug has not been landed yet, plz ignore commnent 4. But obviously, we are trying to update V2 instead of V4.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Please note that this also fails on try now:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a86a93afe649&selectedJob=130580809

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

Ups sorry, that is bug 1385613. I accidentally triggered artifact builds.

Comment 8

[Thomas Nguyen (:tnguyen)]

I am not sure what's happening, but I see in the log, the preferences original is "*-proto" then all will be changed to "*-shavar" (6 times of reading prefs).
Basically, "*-shavar" is defined in all.js but then be overridden in firefox.js. I still don't know why they are rolled back to *-shavar

Comment 9

[François Marier [:francois]]

(In reply to Thomas Nguyen[:tnguyen] ni plz from comment #8)
> I am not sure what's happening, but I see in the log, the preferences
> original is "*-proto" then all will be changed to "*-shavar" (6 times of
> reading prefs).
> Basically, "*-shavar" is defined in all.js but then be overridden in
> firefox.js. I still don't know why they are rolled back to *-shavar

Ah, so what we are seeing is the effect of the crash-rate Shield study that launched yesterday (see bug 1377267). We'll have the same problem on release with the gradual roll-out of V4 via Shield (bug 1387651).

There is probably a race condition in the marionette test because of the fact that we start with V4, then later on switch to V2.

A good work-around would be to disable all Shield studies in this test.

More generally though, should we disable Shield studies from all marionette tests? They could easily interfere with other functionality and cause hard-to-diagnose test failures.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

(In reply to François Marier [:francois] from comment #9)
> More generally though, should we disable Shield studies from all marionette
> tests? They could easily interfere with other functionality and cause
> hard-to-diagnose test failures.

I would definitely vote for that! If you have the necessary prefs can you file a Marionette bug? I could get those added for Marionette and geckodriver then.

Comment 11

[Intermittent Failures Robot]

36 failures in 1032 pushes (0.035 failures/push) were associated with this bug in the last 7 days.   

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-beta: 35
* autoland: 1

Platform breakdown:
* linux64-nightly: 9
* windows7-32-nightly: 5
* windows10-64-nightly: 5
* linux32-nightly: 5
* windows10-64-devedition: 4
* macosx64-nightly: 3
* macosx64-devedition: 2
* linux32-devedition: 2
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399311&startday=2017-09-11&endday=2017-09-17&tree=all

Comment 12

[Intermittent Failures Robot]

1 failures in 885 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* autoland: 1

Platform breakdown:
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399311&startday=2017-09-25&endday=2017-10-01&tree=all

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2] (away 10/03 - 10/13)]

With the landing of my patch on bug 1399628 this is fixed now.

Comment 14

[Intermittent Failures Robot]

2 failures in 947 pushes (0.002 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 2

Platform breakdown:
* windows7-32: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399311&startday=2017-10-09&endday=2017-10-15&tree=all

Comment 15

[Intermittent Failures Robot]

1 failures in 912 pushes (0.001 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* mozilla-central: 1

Platform breakdown:
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1399311&startday=2017-10-23&endday=2017-10-29&tree=all