1292728 - Permaorange test_safe_browsing_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | TimeoutException: Timed out after 25.1 seconds with message: Safe Browsing File: mozstd-track-digest256.cache not found!

Status: RESOLVED FIXED

(Testing :: Firefox UI Tests, defect, P1)

Description

[Treeherder Bug Filer]

Filed by: phil [at] philringnalda.com

https://treeherder.mozilla.org/logviewer.html#?job_id=10985076&repo=fx-team

https://queue.taskcluster.net/v1/task/BoEKnScxQriRvIBn-uhdLA/runs/0/artifacts/public%2Flogs%2Flive_backing.log

Comment 1

[Intermittent Failures Robot]

60 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 34
* fx-team: 18
* autoland: 8

Platform breakdown:
* linux64: 60

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292728&startday=2016-08-06&endday=2016-08-06&tree=all

Comment 2

[Intermittent Failures Robot]

349 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* fx-team: 136
* mozilla-inbound: 88
* autoland: 79
* mozilla-central: 46

Platform breakdown:
* linux64: 349

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292728&startday=2016-08-01&endday=2016-08-07&tree=all

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Panos, with the landing of your changes on bug 1258041 you broke our safebrowsing tests permanently. Can you please update the test so it reflects the current behavior of Firefox?

I would suggest you do try builds in the future to check if changes do not break tests. Thanks.

Comment 4

[Carsten Book [:Tomcat]]

can we fix this asap please, thanks!

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Bug 1292728 - Disable test_safe_browsing_initial_download.py due to perma failures caused by bug 1258041.

Review commit: https://reviewboard.mozilla.org/r/69988/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/69988/

Comment 6

[Andreas Tolfsen ❲:ato❳]

Comment on attachment 8778860 [details]
Bug 1292728 - Disable test_safe_browsing_initial_download.py due to perma failures caused by bug 1258041.

https://reviewboard.mozilla.org/r/69988/#review67158

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f0c07fbbb4b5
Disable test_safe_browsing_initial_download.py due to perma failures caused by bug 1258041. r=ato

Comment 8

[Gian-Carlo Pascutto [:gcp]]

What's required to make these SB tests run in tier-1 (so the commit gets backed out, instead of -urrrgh- the test getting disabled)?

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

They will never become tier-1 because we rely on remote test data for this test. This is strictly forbidden for tier-1 level tests.

Comment 10

[Intermittent Failures Robot]

337 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 149
* autoland: 123
* fx-team: 26
* mozilla-central: 24
* try: 15

Platform breakdown:
* linux64: 323
* linux32: 3
* windows8-64: 2
* windows7-64: 2
* windowsxp: 1
* windows8-32: 1
* windows7-32-vm: 1
* windows7-32: 1
* osx-10-9: 1
* osx-10-11: 1
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292728&startday=2016-08-08&endday=2016-08-08&tree=all

Comment 11

[Panos Astithas (he/him) [:past] (please ni?)]

Attachment: Fix broken test test_safe_browsing_initial_download.py after the recent safebrowsing list changes (bug 1292728).

Review commit: https://reviewboard.mozilla.org/r/70322/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/70322/

Comment 12

[Panos Astithas (he/him) [:past] (please ni?)]

This should be enough to fix the test. Let's see if try agrees:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=09531223be4d

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/f0c07fbbb4b5

Comment 14

[Panos Astithas (he/him) [:past] (please ni?)]

Comment on attachment 8779319 [details]
Fix broken test test_safe_browsing_initial_download.py after the recent safebrowsing list changes (bug 1292728).

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/70322/diff/1-2/

Comment 15

[Panos Astithas (he/him) [:past] (please ni?)]

Try was green and I verified that the test finished successfully. I updated the patch to enable the test again.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8779319 [details]
Fix broken test test_safe_browsing_initial_download.py after the recent safebrowsing list changes (bug 1292728).

https://reviewboard.mozilla.org/r/70322/#review67666

Thanks!

::: testing/firefox-ui/tests/functional/security/test_safe_browsing_initial_download.py:33
(Diff revision 2)
>                  "goog-unwanted-shavar.sbstore",
>  
>                  # Tracking Protections
> -                "mozstd-track-digest256.cache",
> -                "mozstd-track-digest256.pset",
> -                "mozstd-track-digest256.sbstore",
> +                "base-track-digest256.cache",
> +                "base-track-digest256.pset",
> +                "base-track-digest256.sbstore",

Would you mind triggering a try build on linux64 for firefox-ui-functional-remote(-e10s) only? I would like the test passing before we land it.

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Panos Astithas [:past] from comment #15)
> Try was green and I verified that the test finished successfully. I updated
> the patch to enable the test again.

The try build you ran was before you enabled the test, so it has not been executed! Please do it again.

Comment 18

[Panos Astithas (he/him) [:past] (please ni?)]

(In reply to Henrik Skupin (:whimboo) from comment #17)
> The try build you ran was before you enabled the test, so it has not been
> executed! Please do it again.

That's not correct, the test was enabled in my tree as the disabling patch hadn't landed in fx-team at that time. You can confirm this by looking for the test name in the raw test output.

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Oh, in this case go ahead and land it. Thanks for clarifying, given that it was not clear.

Comment 20

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f38875b13ca3
Fix broken test test_safe_browsing_initial_download.py after the recent safebrowsing list changes . r=whimboo

Comment 21

[Intermittent Failures Robot]

94 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 79
* try: 8
* fx-team: 7

Platform breakdown:
* linux64: 94

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292728&startday=2016-08-09&endday=2016-08-09&tree=all

Comment 22

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/f38875b13ca3

Comment 23

[Intermittent Failures Robot]

623 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 325
* autoland: 123
* fx-team: 104
* mozilla-central: 42
* try: 29

Platform breakdown:
* linux64: 598
* windows8-64: 4
* windows7-64: 4
* linux32: 4
* windowsxp: 2
* windows8-32: 2
* windows7-32: 2
* osx-10-9: 2
* osx-10-11: 2
* osx-10-10: 2
* windows7-32-vm: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292728&startday=2016-08-08&endday=2016-08-14&tree=all

Comment 24

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The patch on bug 1258041 got backported to mozilla-aurora now, but this fix has not been incorporated! :( We need to get this onto aurora ASAP to stop the perma orange. Thanks.

Comment 25

[Ryan VanderMeulen (PTO, back July 27)]

https://hg.mozilla.org/releases/mozilla-aurora/rev/b7b4939fdff3

Comment 26

[Intermittent Failures Robot]

61 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-aurora: 61

Platform breakdown:
* linux64: 22
* windows8-64: 7
* windows7-64: 7
* osx-10-9: 4
* osx-10-11: 4
* osx-10-10: 4
* linux32: 4
* windowsxp: 3
* windows8-32: 3
* windows7-32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292728&startday=2016-08-18&endday=2016-08-18&tree=all

Comment 27

[Intermittent Failures Robot]

72 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-aurora: 72

Platform breakdown:
* linux64: 33
* windows8-64: 7
* windows7-64: 7
* osx-10-9: 4
* osx-10-11: 4
* osx-10-10: 4
* linux32: 4
* windowsxp: 3
* windows8-32: 3
* windows7-32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1292728&startday=2016-08-15&endday=2016-08-21&tree=all