Closed Bug 1292728 Opened 7 years ago Closed 7 years ago

Permaorange test_safe_browsing_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | TimeoutException: Timed out after 25.1 seconds with message: Safe Browsing File: mozstd-track-digest256.cache not found!

Categories

(Testing :: Firefox UI Tests, defect, P1)

Product:
Testing
Component:
Firefox UI Tests
Version:
Version 3
Type:
defect

Tracking

(firefox50 fixed, firefox51 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla51
Iteration:
51.1 - Aug 15
Tracking Flags:
Tracking Status
firefox50 --- fixed
firefox51 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: past)

References

Details

(Keywords: intermittent-failure, regression, Whiteboard: [fxprivacy])

Attachments

(2 files)

Bug 1292728 - Disable test_safe_browsing_initial_download.py due to perma failures caused by bug 1258041.
58 bytes, text/x-review-board-request
ato
: review+
Details
Fix broken test test_safe_browsing_initial_download.py after the recent safebrowsing list changes (bug 1292728).
58 bytes, text/x-review-board-request
whimboo
: review+
Details
Blocks: 1258041
Flags: needinfo?(past)
Keywords: regression
Panos, with the landing of your changes on bug 1258041 you broke our safebrowsing tests permanently. Can you please update the test so it reflects the current behavior of Firefox?

I would suggest you do try builds in the future to check if changes do not break tests. Thanks.
Flags: needinfo?(past)
Flags: needinfo?(past)
can we fix this asap please, thanks!
Comment on attachment 8778860 [details]
Bug 1292728 - Disable test_safe_browsing_initial_download.py due to perma failures caused by bug 1258041.

https://reviewboard.mozilla.org/r/69988/#review67158
Attachment #8778860 - Flags: review?(ato) → review+
Pushed by hskupin@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f0c07fbbb4b5
Disable test_safe_browsing_initial_download.py due to perma failures caused by bug 1258041. r=ato
What's required to make these SB tests run in tier-1 (so the commit gets backed out, instead of -urrrgh- the test getting disabled)?
They will never become tier-1 because we rely on remote test data for this test. This is strictly forbidden for tier-1 level tests.
Assignee: nobody → past
Status: NEW → ASSIGNED
Flags: needinfo?(past)
Priority: -- → P1
Whiteboard: [fxprivacy]
This should be enough to fix the test. Let's see if try agrees:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=09531223be4d
Try was green and I verified that the test finished successfully. I updated the patch to enable the test again.
Comment on attachment 8779319 [details]
Fix broken test test_safe_browsing_initial_download.py after the recent safebrowsing list changes (bug 1292728).

https://reviewboard.mozilla.org/r/70322/#review67666

Thanks!

::: testing/firefox-ui/tests/functional/security/test_safe_browsing_initial_download.py:33
(Diff revision 2)
>                  "goog-unwanted-shavar.sbstore",
>  
>                  # Tracking Protections
> -                "mozstd-track-digest256.cache",
> -                "mozstd-track-digest256.pset",
> -                "mozstd-track-digest256.sbstore",
> +                "base-track-digest256.cache",
> +                "base-track-digest256.pset",
> +                "base-track-digest256.sbstore",

Would you mind triggering a try build on linux64 for firefox-ui-functional-remote(-e10s) only? I would like the test passing before we land it.
Attachment #8779319 - Flags: review?(hskupin) → review+
(In reply to Panos Astithas [:past] from comment #15)
> Try was green and I verified that the test finished successfully. I updated
> the patch to enable the test again.

The try build you ran was before you enabled the test, so it has not been executed! Please do it again.
(In reply to Henrik Skupin (:whimboo) from comment #17)
> The try build you ran was before you enabled the test, so it has not been
> executed! Please do it again.

That's not correct, the test was enabled in my tree as the disabling patch hadn't landed in fx-team at that time. You can confirm this by looking for the test name in the raw test output.
Oh, in this case go ahead and land it. Thanks for clarifying, given that it was not clear.
Pushed by pastithas@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f38875b13ca3
Fix broken test test_safe_browsing_initial_download.py after the recent safebrowsing list changes . r=whimboo
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Keywords: leave-open
Resolution: --- → FIXED
Target Milestone: --- → mozilla51
Iteration: --- → 51.1 - Aug 15
The patch on bug 1258041 got backported to mozilla-aurora now, but this fix has not been incorporated! :( We need to get this onto aurora ASAP to stop the perma orange. Thanks.
status-firefox50: --- → affected
status-firefox51: --- → fixed
Whiteboard: [fxprivacy] → [checkin-needed-aurora][fxprivacy]
https://hg.mozilla.org/releases/mozilla-aurora/rev/b7b4939fdff3
status-firefox50: affected → fixed
Whiteboard: [checkin-needed-aurora][fxprivacy] → [fxprivacy]
You need to log in before you can comment on or make changes to this bug.