TEST-UNEXPECTED-ERROR | test_safe_browsing_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages | NoSuchElementException: NoSuchElementException: Unable to locate element: reportButton

RESOLVED WORKSFORME

Status

RESOLVED WORKSFORME
2 years ago
a year ago

People

(Reporter: whimboo, Unassigned)

Tracking

({intermittent-failure})

49 Branch
intermittent-failure
Points:
---

Firefox Tracking Flags

(firefox49 affected, firefox50 affected)

Details

(Reporter)

Description

2 years ago
https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=Firefox%20UI&filter-tier=2&filter-tier=3&fromchange=eeb73478d98476877f2035f0c330fdc89e361803&selectedJob=28449773

 15:38:09     INFO - TEST-UNEXPECTED-ERROR | test_safe_browsing_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages | NoSuchElementException: NoSuchElementException: Unable to locate element: reportButton

 15:38:09     INFO - Traceback (most recent call last):

 15:38:09     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 344, in run

 15:38:09     INFO -     testMethod()

 15:38:09     INFO -   File "/home/worker/workspace/build/tests/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py", line 58, in test_warning_pages

 15:38:09     INFO -     self.check_report_button(unsafe_page)

 15:38:09     INFO -   File "/home/worker/workspace/build/tests/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py", line 82, in check_report_button

15:38:09 INFO - button = self.marionette.find_element(By.ID, "reportButton")
(Reporter)

Comment 1

2 years ago
Single failure only. Lets reopen if it appears again.
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → WORKSFORME
(Reporter)

Comment 2

2 years ago
Happened on mozilla-aurora yesterday. Reopening.
Status: RESOLVED → REOPENED
status-firefox50: --- → affected
Resolution: WORKSFORME → ---
(Reporter)

Comment 3

2 years ago
The problem here is the following lines:

https://dxr.mozilla.org/mozilla-central/rev/12637ae351d64ecbf6b74cdbf26d7eb24ac0f659/testing/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py#56-60

We load the unsafe page and wait an additional 1s for the about:blocked page to be there. Looks like this is not going to happen, or is delayed. I would propose that we instead remove the call to sleep(1) and change the following assert to an Wait().until() with a page load timeout similar to bug 1321480.
6 failures in 563 pushes (0.011 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* autoland: 4
* mozilla-inbound: 2

Platform breakdown:
* linux64: 6

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1275219&startday=2017-01-02&endday=2017-01-08&tree=all
7 failures in 722 pushes (0.01 failures/push) were associated with this bug in the last 7 days.  

Repository breakdown:
* autoland: 5
* mozilla-inbound: 2

Platform breakdown:
* linux64: 6
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1275219&startday=2017-01-09&endday=2017-01-15&tree=all
14 failures in 836 pushes (0.017 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* mozilla-inbound: 6
* autoland: 3
* try: 2
* mozilla-beta: 2
* mozilla-central: 1

Platform breakdown:
* linux64: 7
* linux32: 7

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1275219&startday=2017-02-06&endday=2017-02-12&tree=all
8 failures in 812 pushes (0.01 failures/push) were associated with this bug in the last 7 days.  
Repository breakdown:
* autoland: 3
* graphics: 2
* try: 1
* mozilla-inbound: 1
* mozilla-aurora: 1

Platform breakdown:
* linux32: 5
* linux64: 3

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1275219&startday=2017-02-20&endday=2017-02-26&tree=all
29 failures in 783 pushes (0.037 failures/push) were associated with this bug in the last 7 days. 

This is the #44 most frequent failure this week. 
Repository breakdown:
* autoland: 14
* mozilla-central: 7
* try: 3
* mozilla-inbound: 2
* oak: 1
* mozilla-esr52: 1
* mozilla-aurora: 1

Platform breakdown:
* linux32: 17
* linux64: 11
* windows7-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1275219&startday=2017-02-27&endday=2017-03-05&tree=all
this picked up a lot in the last week, not enough to get concerned with, but enough to keep an eye on this.
(Reporter)

Comment 10

2 years ago
There seem to be two different situations here:

1) The requested URL cannot be loaded and we have a strange notification bar content showing up: https://public-artifacts.taskcluster.net/Dc2aY7RnRdqZm18OI0YPow/0/public/test_info//report.html

2) The page https://www.itisatrap.org/firefox/unwanted.html gets loaded but doesn't get marked as faulty: https://public-artifacts.taskcluster.net/YfpvCYXCT_eu0Ngjy4QUkg/0/public/test_info//report.html

So 1) could be a networking issue and might not happen that often. I don't have the time now to check all the reports. But 2) looks suspicious and I feel that the necessary safebrowsing blacklist items are not available yet. Francois, are those built-in for itisatrap.org, or do we also have to download those? Please remember that we do not wait for all data downloaded as done by initial download, and that the latter might have to be part of setUp of all safebrowsing related tests.
Flags: needinfo?(francois)
(In reply to Henrik Skupin (:whimboo) from comment #10)
> Francois, are those built-in for itisatrap.org, or do we also have to
> download those? Please remember that we do not wait for all data downloaded
> as done by initial download, and that the latter might have to be part of
> setUp of all safebrowsing related tests.

Yes, the itisatrap.org URLs are built-in (no need to download them) and are loading within a few seconds of the browser starting up.

Maybe there's a race condition where the tests get run immediately at startup and before we load the test entries from a slow disk?
Flags: needinfo?(francois)
(Reporter)

Comment 12

2 years ago
We do not reset the profile and restart Firefox for those tests, so they are run somewhere at the end of the test run. I doubt that the above is a problem here.
(Reporter)

Comment 13

2 years ago
(In reply to François Marier [:francois] from comment #11)
> Maybe there's a race condition where the tests get run immediately at
> startup and before we load the test entries from a slow disk?

Is there a way to check if the entries have already been added to the internal database? When I had another look at this test we have a plain `sleep(3)` in `setUp` which can indeed be very racy.
Flags: needinfo?(francois)
(In reply to Henrik Skupin (:whimboo) from comment #13)
> (In reply to François Marier [:francois] from comment #11)
> > Maybe there's a race condition where the tests get run immediately at
> > startup and before we load the test entries from a slow disk?
> 
> Is there a way to check if the entries have already been added to the
> internal database? When I had another look at this test we have a plain
> `sleep(3)` in `setUp` which can indeed be very racy.

I'm not 100% sure, but I think that they are written to disk (~/.cache/mozilla/firefox/XXXX/safebrowsing/test-*) after they are loaded in memory. So if you check for the existence of the built-in test lists, you should be able to reliably detect that the lists are loaded.

I think my comment above was wrong, the entries aren't loaded from disk, they are added manually (hard-coded entries) and then written out to disk.
Flags: needinfo?(francois)
(Reporter)

Comment 15

2 years ago
This might indicate an improvement. Sadly I have to say that I'm not able to work on anything for the time being. If it turns out to get a larger issue, someone else has to fix it.
I think this might be a dup of bug 1400992?
Blocks: 1328634
Flags: needinfo?(hskupin)
(Reporter)

Comment 17

a year ago
Not really, the patch on bug 1400992 simply changed the name of the button. This failure is around way longer.

But I would suggest to close this bug given that the failure didn't appear in the last 7 months.
Status: REOPENED → RESOLVED
Last Resolved: 2 years agoa year ago
Flags: needinfo?(hskupin)
Resolution: --- → WORKSFORME
29 failures in 199 pushes (0.146 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* try: 29

Platform breakdown:
* linux64: 11
* osx-10-10: 8
* linux32: 5
* windows7-32: 3
* windows10-64: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1275219&startday=2017-09-21&endday=2017-09-21&tree=all
58 failures in 943 pushes (0.062 failures/push) were associated with this bug in the last 7 days. 

This is the #31 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. **  

Repository breakdown:
* try: 58

Platform breakdown:
* linux64: 27
* osx-10-10: 12
* windows7-32: 7
* windows10-64: 7
* linux32: 5

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1275219&startday=2017-09-18&endday=2017-09-24&tree=all
You need to log in before you can comment on or make changes to this bug.