1252037 - TEST-UNEXPECTED-ERROR | test_safe_browsing_notification.py TestSafeBrowsingNotificationBar.test_notification_bar | MarionetteException: MarionetteException: Property not found: safebrowsing.notAForgeryButton.label

Status: RESOLVED FIXED

(Testing :: Firefox UI Tests, defect, P1)

Description

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Since our Fx UI Tests are getting executed via TaskCluster we have this new kind of test failure. I have not seen it on Try! It started with the landing on inbound.

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=Firefox%20UI&filter-tier=2&filter-tier=3&selectedJob=22620661

 11:23:57     INFO - TEST-UNEXPECTED-ERROR | test_safe_browsing_notification.py TestSafeBrowsingNotificationBar.test_notification_bar | MarionetteException: MarionetteException: Property not found: safebrowsing.notAForgeryButton.label

 11:23:57     INFO - Traceback (most recent call last):

 11:23:57     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 344, in run

 11:23:57     INFO -     testMethod()

 11:23:57     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/firefox_ui_tests/functional/security/test_safe_browsing_notification.py", line 73, in test_notification_bar

 11:23:57     INFO -     self.check_not_badware_button(button_property, report_page)

 11:23:57     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/firefox_ui_tests/functional/security/test_safe_browsing_notification.py", line 106, in check_not_badware_button

 11:23:57     INFO -     label = self.browser.get_property(button_property)

 11:23:57     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/firefox_puppeteer/ui/windows.py", line 323, in get_property

 11:23:57     INFO -     return self._l10n.get_property(self.properties, property_id)

 11:23:57     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/firefox_puppeteer/api/l10n.py", line 90, in get_property

11:23:57 INFO - raise MarionetteException('Property not found: %s' % property_id) 

Not sure yet what's special or different in those taskcluster tasks, but this is suspicious given the permanent failure.

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

So the property definitely exists:

https://dxr.mozilla.org/mozilla-central/source/browser/locales/en-US/chrome/browser/browser.properties#407

And we import the properties file in the browser window class here:

https://dxr.mozilla.org/mozilla-central/source/testing/puppeteer/firefox/firefox_puppeteer/ui/browser/window.py#37

So this feels very strange to me.

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Thanks to Flod who pointed me to bug 1245992 which actually caused this test failure.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Attachment: MozReview Request: Bug 1252037 - Adjust safe_browsing_notification test for property changes. r?automatedtester

Review commit: https://reviewboard.mozilla.org/r/37119/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/37119/

Comment 4

[David Burns :automatedtester]

Comment on attachment 8724702 [details]
MozReview Request: Bug 1252037 - Adjust safe_browsing_notification test for property changes. r?automatedtester

https://reviewboard.mozilla.org/r/37119/#review33671

Comment 5

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b26f4f428a14

Comment 6

[Intermittent Failures Robot]

63 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-inbound: 52
* mozilla-central: 11

Platform breakdown:
* linux64: 55
* windows8-64: 1
* windows8-32: 1
* windows7-64: 1
* osx-10-9: 1
* osx-10-6: 1
* osx-10-11: 1
* osx-10-10: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1252037&startday=2016-02-29&endday=2016-02-29&tree=all

Comment 7

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/b26f4f428a14

Comment 8

[Intermittent Failures Robot]

17 automation job failures were associated with this bug yesterday.

Repository breakdown:
* mozilla-central: 17

Platform breakdown:
* linux64: 5
* windows8-64: 2
* windows7-64: 2
* windowsxp: 1
* windows8-32: 1
* windows7-32: 1
* osx-10-9: 1
* osx-10-6: 1
* osx-10-11: 1
* osx-10-10: 1
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1252037&startday=2016-03-01&endday=2016-03-01&tree=all

Comment 9

[Intermittent Failures Robot]

83 automation job failures were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-inbound: 52
* mozilla-central: 31

Platform breakdown:
* linux64: 60
* windows8-64: 4
* windows7-64: 4
* windowsxp: 2
* windows8-32: 2
* osx-10-9: 2
* osx-10-6: 2
* osx-10-11: 2
* osx-10-10: 2
* linux32: 2
* windows7-32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1252037&startday=2016-02-29&endday=2016-03-06&tree=all