1280878 - Intermittent test_safe_browsing_notification.py TestSafeBrowsingNotificationBar.test_notification_bar | TimeoutException: Timed out after 30.1 seconds

Status: RESOLVED FIXED

(Testing :: Firefox UI Tests, defect)

Description

[Henrik Skupin [:whimboo][⌚️UTC+2]]

https://treeherder.mozilla.org/logviewer.html#?job_id=4110551&repo=mozilla-central

 04:45:18     INFO - TEST-UNEXPECTED-ERROR | test_safe_browsing_notification.py TestSafeBrowsingNotificationBar.test_notification_bar | TimeoutException: TimeoutException: Timed out after 30.0 seconds
 04:45:18     INFO - Traceback (most recent call last):
 04:45:18     INFO -   File "/Users/mozauto/jenkins/workspace/mozilla-central_functional/build/venv/lib/python2.7/site-packages/marionette/marionette_test.py", line 344, in run
 04:45:18     INFO -     testMethod()
 04:45:18     INFO -   File "/Users/mozauto/jenkins/workspace/mozilla-central_functional/build/tests/firefox-ui/tests/functional/security/test_safe_browsing_notification.py", line 73, in test_notification_bar
 04:45:18     INFO -     self.check_not_badware_button(button_property, report_page)
 04:45:18     INFO -   File "/Users/mozauto/jenkins/workspace/mozilla-central_functional/build/tests/firefox-ui/tests/functional/security/test_safe_browsing_notification.py", line 113, in check_not_badware_button
 04:45:18     INFO -     lambda mn: report_page in mn.get_url())
 04:45:18     INFO -   File "/Users/mozauto/jenkins/workspace/mozilla-central_functional/build/venv/lib/python2.7/site-packages/marionette_driver/wait.py", line 144, in until
 04:45:18     INFO -     cause=last_exc)
04:45:18 INFO - TEST-INFO took 44851ms

Looks like we fail to load the report page.

Comment 1

[Intermittent Failures Robot]

16 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-beta: 8
* mozilla-aurora: 8

Platform breakdown:
* windows8-64: 5
* windows7-64: 5
* windows8-32: 2
* windows7-32: 2
* windowsxp: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-07-22&endday=2016-07-22&tree=all

Comment 2

[Intermittent Failures Robot]

32 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-aurora: 16
* mozilla-beta: 8
* mozilla-inbound: 3
* mozilla-central: 3
* autoland: 2

Platform breakdown:
* windows7-64: 8
* windows8-64: 7
* linux64: 5
* windows7-32: 3
* windows8-32: 2
* osx-10-11: 2
* linux32: 2
* windowsxp: 1
* osx-10-9: 1
* osx-10-10: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-07-18&endday=2016-07-24&tree=all

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Fixed by the proxy changes on bug 1289697.

Comment 4

[Intermittent Failures Robot]

33 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-release: 23
* mozilla-aurora: 7
* mozilla-central: 3

Platform breakdown:
* windows8-64: 8
* windows7-64: 6
* windowsxp: 4
* windows7-32: 3
* osx-10-10: 3
* windows8-32: 2
* osx-10-6: 2
* osx-10-11: 2
* linux32: 2
* osx-10-9: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-07-25&endday=2016-07-31&tree=all

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

It still randomly fails due to networking issues. Lets keep it open to get more data.

Comment 6

[Intermittent Failures Robot]

11 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 5
* mozilla-beta: 3
* mozilla-aurora: 3

Platform breakdown:
* linux64: 6
* windows7-64: 3
* windows8-64: 1
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-08-29&endday=2016-09-04&tree=all

Comment 7

[Intermittent Failures Robot]

8 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 7
* mozilla-aurora: 1

Platform breakdown:
* linux64: 7
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-09-12&endday=2016-09-18&tree=all

Comment 8

[Intermittent Failures Robot]

7 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 4
* mozilla-inbound: 1
* mozilla-aurora: 1
* fx-team: 1

Platform breakdown:
* linux64: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-10-10&endday=2016-10-16&tree=all

Comment 9

[Intermittent Failures Robot]

15 automation job failures were associated with this bug yesterday.

Repository breakdown:
* try: 8
* autoland: 4
* mozilla-inbound: 3

Platform breakdown:
* linux64: 15

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-10-25&endday=2016-10-25&tree=all

Comment 10

[Joel Maher ( :jmaher ) (UTC -8)]

:whimboo, yesterday this intermittent spiked:
https://brasstacks.mozilla.com/orangefactor/index.html?display=Bug&bugid=1280878&startday=2016-10-19&endday=2016-10-26&tree=trunk

can you take a look at this?  it is one of the top 5 intermittents.  It is split between different buildtypes for linux64 and appears to be split between opt-firefox-ui-functional-remote-e10s and opt-firefox-ui-functional-remote.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

The failing line is here:

https://dxr.mozilla.org/mozilla-beta/source/testing/firefox-ui/tests/functional/security/test_safe_browsing_notification.py#113

It checks if the correct report page has been opened, and fails in that:

report page:
www.google.com/safebrowsing/report_error

current URL:
https://safebrowsing.google.com/safebrowsing/report_error/?tpl=mozilla&hl=en-US&url=https%3A%2F%2Fwww.itisatrap.org%2Ffirefox%2Fits-a-trap.html

So it looks like that Google actually has moved that page from the www subdomain to safebrowsing. I would suggest we remove the subdomain from the check and only check for the domain and fragment.

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Attachment: Bug 1280878 - Update safebrowing_notification test for changed Google URL.

Review commit: https://reviewboard.mozilla.org/r/88544/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/88544/

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Because autolander is currently not working I pushed the try build myself:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f17fd98707e5af2294714e59c80c899ed562a6bc

Comment 14

[François Marier [:francois]]

Comment on attachment 8804626 [details]
Bug 1280878 - Update safebrowing_notification test for changed Google URL.

https://reviewboard.mozilla.org/r/88544/#review87700

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/aff9d3eb8975
Update safebrowing_notification test for changed Google URL. r=francois

Comment 16

[Intermittent Failures Robot]

288 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 128
* mozilla-aurora: 50
* autoland: 35
* mozilla-central: 33
* try: 24
* fx-team: 12
* mozilla-beta: 6

Platform breakdown:
* linux64: 250
* windows8-64: 7
* windows7-64: 7
* osx-10-9: 4
* osx-10-11: 4
* osx-10-10: 4
* windowsxp: 3
* windows8-32: 3
* windows7-32: 3
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-10-26&endday=2016-10-26&tree=all

Comment 17

[Phil Ringnalda (:philor)]

https://hg.mozilla.org/mozilla-central/rev/aff9d3eb8975

Comment 18

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr45/rev/794cabe7579d

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/88cbb2726969

Comment 20

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/16e7406cee3d

Comment 21

[Intermittent Failures Robot]

30 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-aurora: 16
* mozilla-release: 6
* mozilla-beta: 6
* mozilla-inbound: 2

Platform breakdown:
* linux64: 30

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-10-27&endday=2016-10-27&tree=all

Comment 22

[Intermittent Failures Robot]

387 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 147
* mozilla-aurora: 66
* mozilla-central: 44
* autoland: 41
* try: 36
* mozilla-release: 24
* mozilla-beta: 17
* fx-team: 12

Platform breakdown:
* linux64: 339
* windows8-64: 9
* windows7-64: 9
* osx-10-9: 5
* osx-10-11: 5
* osx-10-10: 5
* windowsxp: 4
* windows8-32: 4
* windows7-32: 4
* linux32: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-10-24&endday=2016-10-30&tree=all

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #20)
> https://hg.mozilla.org/releases/mozilla-beta/rev/16e7406cee3d

Please note that the latest tests for Firefox 47.0.2 where broken because this patch wasn't part of mozilla-release until the merge from yesterday:

https://treeherder.mozilla.org/#/jobs?repo=mozilla-release&revision=d7f98b92fdeca9a662206667c68ec41a1476bdb7&selectedJob=318068&filter-searchStr=firefox%20functional&filter-tier=1&filter-tier=2&filter-tier=3

Comment 24

[Intermittent Failures Robot]

108 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-release: 108

Platform breakdown:
* windows8-64: 18
* windows7-64: 18
* windowsxp: 9
* windows8-32: 9
* windows7-32: 9
* osx-10-9: 9
* osx-10-11: 9
* osx-10-10: 9
* linux64: 9
* linux32: 9

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-11-01&endday=2016-11-01&tree=all

Comment 25

[Intermittent Failures Robot]

116 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-release: 116

Platform breakdown:
* windows8-64: 18
* windows7-64: 18
* linux64: 17
* windowsxp: 9
* windows8-32: 9
* windows7-32: 9
* osx-10-9: 9
* osx-10-11: 9
* osx-10-10: 9
* linux32: 9

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1280878&startday=2016-10-31&endday=2016-11-06&tree=all