1484517 - Intermittent testing\firefox-ui\tests\functional\safebrowsing\test_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages | AttributeError: 'NoneType' object has no attribute 'click'

Status: RESOLVED DUPLICATE of bug 1482029

(Testing :: Firefox UI Tests, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: apavel [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=194748404&repo=autoland

https://queue.taskcluster.net/v1/task/Og9k8v6MTWiU3lsJ5jrhXg/runs/0/artifacts/public/logs/live_backing.log

11:13:13     INFO -  ###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost
11:13:13     INFO -  [Parent 7316, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 346
11:13:13     INFO -  [Parent 7316, Gecko_IOThread] WARNING: pipe error: 109: file z:/build/build/src/ipc/chromium/src/chrome/common/ipc_channel_win.cc, line 346
11:13:14     INFO -  1534677194482	Marionette	DEBUG	Received observer notification xpcom-will-shutdown
11:13:14     INFO -  1534677194482	Marionette	INFO	Stopped listening on port 2828
11:13:14     INFO -  1534677194482	Marionette	DEBUG	Remote service is inactive
11:13:15  WARNING - Failed to gather test failure debug: Process has been unexpectedly closed (Exit code: 0) (Reason: No data received over socket)
11:13:15     INFO - TEST-UNEXPECTED-ERROR | testing\firefox-ui\tests\functional\safebrowsing\test_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages | AttributeError: 'NoneType' object has no attribute 'click'
11:13:15     INFO - Traceback (most recent call last):
11:13:15     INFO -   File "Z:\task_1534676832\build\venv\lib\site-packages\marionette_harness\marionette_test\testcases.py", line 159, in run
11:13:15     INFO -     testMethod()
11:13:15     INFO -   File "Z:\task_1534676832\build\tests\firefox-ui\tests\testing\firefox-ui\tests\functional\safebrowsing\test_warning_pages.py", line 63, in test_warning_pages
11:13:15     INFO -     self.check_report_link(unsafe_page)
11:13:15     INFO -   File "Z:\task_1534676832\build\tests\firefox-ui\tests\testing\firefox-ui\tests\functional\safebrowsing\test_warning_pages.py", line 90, in check_report_link
11:13:15     INFO -     link.click()
11:13:15     INFO - TEST-INFO took 22102ms
11:13:15    ERROR - test_end for testing\firefox-ui\tests\functional\safebrowsing\test_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages logged while not in progress. Logged with data: {"status": "ERROR", "extra": {"class_name": "test_warning_pages.TestSafeBrowsingWarningPages", "method_name": "test_warning_pages"}, "expected": "PASS", "test": "testing\\firefox-ui\\tests\\functional\\safebrowsing\\test_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages", "message": "MarionetteException: Please start a session", "stack": "Traceback (most recent call last):\n  File \"Z:\\task_1534676832\\build\\venv\\lib\\site-packages\\marionette_harness\\marionette_test\\testcases.py\", line 190, in run\n    self.tearDown()\n  File \"Z:\\task_1534676832\\build\\tests\\firefox-ui\\tests\\testing\\firefox-ui\\tests\\functional\\safebrowsing\\test_warning_pages.py\", line 48, in tearDown\n    super(TestSafeBrowsingWarningPages, self).tearDown()\n  File \"Z:\\task_1534676832\\build\\venv\\lib\\site-packages\\firefox_puppeteer\\mixins.py\", line 97, in tearDown\n    self.marionette.set_context('chrome')\n  File \"Z:\\task_1534676832\\build\\venv\\lib\\site-packages\\marionette_driver\\marionette.py\", line 1426, in set_context\n    {\"value\": context})\n  File \"Z:\\task_1534676832\\build\\venv\\lib\\site-packages\\marionette_driver\\decorators.py\", line 26, in _\n    return func(*args, **kwargs)\n  File \"Z:\\task_1534676832\\build\\venv\\lib\\site-packages\\marionette_driver\\marionette.py\", line 745, in _send_message\n    raise errors.MarionetteException(\"Please start a session\")\n"}

Comment 1

[Intermittent Failures Robot]

1 failures in 2672 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* autoland: 1

Platform breakdown:
* windows10-64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1484517&startday=2018-08-13&endday=2018-08-19&tree=all

Comment 2

[Henrik Skupin [:whimboo][⌚️UTC+2]]

There as a pipe error and Firefox just closed:

> 11:13:14     INFO -  1534677194482	Marionette	DEBUG	Received observer notification xpcom-will-shutdown
> 11:13:14     INFO -  1534677194482	Marionette	INFO	Stopped listening on port 2828
> 11:13:14     INFO -  1534677194482	Marionette	DEBUG	Remote service is inactive
> 11:13:15  WARNING - Failed to gather test failure debug: Process has been unexpectedly closed (Exit code: 0) (Reason: No data received over socket)

Comment 3

[Andreea Pavel [:apavel]]

Retriggered: https://tinyurl.com/y75aaymk

Comment 4

[Eliza Balazs [:ebalazs_]]

The retrigger result is not very clear:

https://treeherder.mozilla.org/#/jobs?repo=autoland&filter-searchStr=windows%20firefox%20functional%20tests%20(remote)%20with%20e10s%20fxfn-r-e10s(en-us)&tochange=15cca5b18ec422dbc5b7f893f07a7410ec18300f&fromchange=b841b6fa93f18ba50ce3ee73e3203f58ca90cecb

Could this be related to https://bugzilla.mozilla.org/show_bug.cgi?id=1482029 ?

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

I agree it isn't very clear, there is a lot of data here.  I don't see bug 1482029 in the url linked above, I see what looks to be an increase in failures on Bug 1484461, but that could easily be misleading.  I suspect we just need the test author to work on fixing this test.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So I'm fairly sure that any of the classified failures as shown by Orange Factor for this bug are wrong! They are all related to bug 1482029. Not sure why everyone classifies the assertion against this bug, especially when the bug summary is not visible in the failure summer on Treeherder at all. This bug is not a catch all.

Sebastian, can you please make sure that this false classification stops? Thanks.

Comment 7

[Andreea Pavel [:apavel]]

(In reply to Henrik Skupin (:whimboo) from comment #6)
> So I'm fairly sure that any of the classified failures as shown by Orange
> Factor for this bug are wrong! They are all related to bug 1482029. Not sure
> why everyone classifies the assertion against this bug, especially when the
> bug summary is not visible in the failure summer on Treeherder at all. This
> bug is not a catch all.
> 
> Sebastian, can you please make sure that this false classification stops?
> Thanks.

Hi. When a test fails on 2 or more files (has 2 or more failure lines that seem unrelated) we usually classify it with all the bugs suggested. Again, if they are not related, meaning the failure is not on the same file.

I guess that's what we did here too. Ayrx explained the difference here and we'll try as much as possible not to missclassify any other bugs. 

We apologize for the confusion regarding this failure.

Comment 8

[Intermittent Failures Robot]

23 failures in 2568 pushes (0.009 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 4
* try: 3
* mozilla-inbound: 7
* autoland: 9

Platform breakdown:
* linux64: 1
* windows10-64: 19
* linux32: 1
* windows7-32-msvc: 2

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1484517&startday=2018-08-20&endday=2018-08-26&tree=all

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to Andreea Pavel [:apavel] from comment #7)
> Hi. When a test fails on 2 or more files (has 2 or more failure lines that
> seem unrelated) we usually classify it with all the bugs suggested. Again,
> if they are not related, meaning the failure is not on the same file.

That's not what happened in most of the cases. See for example https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=e4e2245fc1428c33db179ca2f242b922ad988d79&selectedJob=196015814. Only this bug has been classified.

Also note that the assertion right before was raised nearly at the same time, which means that it has a side-effect on the test.

So as long as we keep classifying the real underlying problem to raise it's priority through OF I'm fine. 

I will mark the dependency to bug 1482029 now.

Comment 10

[Andreea Pavel [:apavel]]

(In reply to Henrik Skupin (:whimboo) from comment #9)
> (In reply to Andreea Pavel [:apavel] from comment #7)
> > Hi. When a test fails on 2 or more files (has 2 or more failure lines that
> > seem unrelated) we usually classify it with all the bugs suggested. Again,
> > if they are not related, meaning the failure is not on the same file.
> 
> That's not what happened in most of the cases. See for example
> https://treeherder.mozilla.org/#/
> jobs?repo=autoland&revision=e4e2245fc1428c33db179ca2f242b922ad988d79&selected
> Job=196015814. Only this bug has been classified.
> 
> Also note that the assertion right before was raised nearly at the same
> time, which means that it has a side-effect on the test.
> 
> So as long as we keep classifying the real underlying problem to raise it's
> priority through OF I'm fine. 
> 
> I will mark the dependency to bug 1482029 now.

The example you gave me, is one classified by Aryx, which can be a correction from when he looked over what we did.

Comment 11

[Intermittent Failures Robot]

41 failures in 2545 pushes (0.016 failures/push) were associated with this bug in the last 7 days.

This is the #39 most frequent failure this week.

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* mozilla-central: 16
* autoland: 11
* try: 4
* mozilla-inbound: 10

Platform breakdown:
* windows7-32-msvc: 1
* windows10-64-msvc: 3
* windows10-64: 33
* windows7-32: 2
* linux32: 1
* linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1484517&startday=2018-08-27&endday=2018-09-02&tree=all

Comment 12

[Henrik Skupin [:whimboo][⌚️UTC+2]]

No more failures since bug 1482029 was fixed.

Comment 13

[Intermittent Failures Robot]

5 failures in 2667 pushes (0.002 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:
* mozilla-central: 1
* try: 2
* autoland: 2

Platform breakdown:
* windows10-64: 5

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1484517&startday=2018-09-03&endday=2018-09-09&tree=all