1336895 - permanent test_safe_browsing_initial_download.py TestSafeBrowsingInitialDownload.test_safe_browsing_initial_download | AssertionError: Items in the second set but not the first:

Status: RESOLVED DUPLICATE of bug 1333303

(Testing :: Firefox UI Tests, defect)

Description

[Treeherder Bug Filer]

Filed by: ihsiao [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=74612364&repo=autoland

https://queue.taskcluster.net/v1/task/PUhWxQD8RrWHY_BsYmRqtA/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Iris Hsiao [:ihsiao]]

Kirk,
This is a permanent failure from https://hg.mozilla.org/integration/autoland/rev/c3360defc879, could you help to check?

Comment 2

[Carsten Book [:Tomcat]]

indeed, henrik do you know a quick fix or shall we disable this test ?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

It's a regression and as such should block bug 1333483. Updating the dependencies.

I just reviewed the patch on bug 1333303 and pushed to autoland. But not sure how much it will help. Maybe we wait for its results before skipping it? 

Also Francois is maintaining this test those days. I don't really have the time to check it. So redirecting the ni? request to him.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Landing the patch on bug 1333303 fixed the perma failure. I will still keep the ni? set for feedback from Kevin and Francois.

Comment 5

[Robin Steuber (they/them) [:bytesized]]

I don't really understand why the patch for bug 1333483 would cause the failure in bug 1333303 to occur in nightly. I think Francois will be able to better speak to this.
(It's Kirk btw, not Kevin)

Comment 6

[Intermittent Failures Robot]

86 failures in 117 pushes (0.735 failures/push) were associated with this bug yesterday.  

Repository breakdown:
* autoland: 86

Platform breakdown:
* linux64: 58
* linux32: 28

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1336895&startday=2017-02-06&endday=2017-02-06&tree=all

Comment 7

[François Marier [:francois]]

Bug 1333303 made the test tolerant of new lists added and should fix this issue too.

Comment 8

[Intermittent Failures Robot]

86 failures in 836 pushes (0.103 failures/push) were associated with this bug in the last 7 days. 

This is the #22 most frequent failure this week. 

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **

Repository breakdown:
* autoland: 86

Platform breakdown:
* linux64: 58
* linux32: 28

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1336895&startday=2017-02-06&endday=2017-02-12&tree=all