Closed Bug 1275219 Opened 8 years ago Closed 7 years ago

TEST-UNEXPECTED-ERROR | test_safe_browsing_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages | NoSuchElementException: NoSuchElementException: Unable to locate element: reportButton

Categories

(Testing :: Firefox UI Tests, defect)

Product:
Testing
Component:
Firefox UI Tests
Version:
49 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox49 affected, firefox50 affected)

Status:
RESOLVED WORKSFORME
Tracking Flags:
Tracking Status
firefox49 --- affected
firefox50 --- affected

People

(Reporter: whimboo, Unassigned)

References

Details

(Keywords: intermittent-failure) 

https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&filter-searchStr=Firefox%20UI&filter-tier=2&filter-tier=3&fromchange=eeb73478d98476877f2035f0c330fdc89e361803&selectedJob=28449773

 15:38:09     INFO - TEST-UNEXPECTED-ERROR | test_safe_browsing_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages | NoSuchElementException: NoSuchElementException: Unable to locate element: reportButton

 15:38:09     INFO - Traceback (most recent call last):

 15:38:09     INFO -   File "/home/worker/workspace/build/venv/local/lib/python2.7/site-packages/marionette/marionette_test.py", line 344, in run

 15:38:09     INFO -     testMethod()

 15:38:09     INFO -   File "/home/worker/workspace/build/tests/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py", line 58, in test_warning_pages

 15:38:09     INFO -     self.check_report_button(unsafe_page)

 15:38:09     INFO -   File "/home/worker/workspace/build/tests/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py", line 82, in check_report_button

15:38:09 INFO - button = self.marionette.find_element(By.ID, "reportButton")
Single failure only. Lets reopen if it appears again.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
Happened on mozilla-aurora yesterday. Reopening.
Status: RESOLVED → REOPENED
status-firefox50: --- → affected
Resolution: WORKSFORME → ---
The problem here is the following lines:

https://dxr.mozilla.org/mozilla-central/rev/12637ae351d64ecbf6b74cdbf26d7eb24ac0f659/testing/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py#56-60

We load the unsafe page and wait an additional 1s for the about:blocked page to be there. Looks like this is not going to happen, or is delayed. I would propose that we instead remove the call to sleep(1) and change the following assert to an Wait().until() with a page load timeout similar to bug 1321480.
this picked up a lot in the last week, not enough to get concerned with, but enough to keep an eye on this.
There seem to be two different situations here:

1) The requested URL cannot be loaded and we have a strange notification bar content showing up: https://public-artifacts.taskcluster.net/Dc2aY7RnRdqZm18OI0YPow/0/public/test_info//report.html

2) The page https://www.itisatrap.org/firefox/unwanted.html gets loaded but doesn't get marked as faulty: https://public-artifacts.taskcluster.net/YfpvCYXCT_eu0Ngjy4QUkg/0/public/test_info//report.html

So 1) could be a networking issue and might not happen that often. I don't have the time now to check all the reports. But 2) looks suspicious and I feel that the necessary safebrowsing blacklist items are not available yet. Francois, are those built-in for itisatrap.org, or do we also have to download those? Please remember that we do not wait for all data downloaded as done by initial download, and that the latter might have to be part of setUp of all safebrowsing related tests.
Flags: needinfo?(francois)
(In reply to Henrik Skupin (:whimboo) from comment #10)
> Francois, are those built-in for itisatrap.org, or do we also have to
> download those? Please remember that we do not wait for all data downloaded
> as done by initial download, and that the latter might have to be part of
> setUp of all safebrowsing related tests.

Yes, the itisatrap.org URLs are built-in (no need to download them) and are loading within a few seconds of the browser starting up.

Maybe there's a race condition where the tests get run immediately at startup and before we load the test entries from a slow disk?
Flags: needinfo?(francois)
We do not reset the profile and restart Firefox for those tests, so they are run somewhere at the end of the test run. I doubt that the above is a problem here.
(In reply to François Marier [:francois] from comment #11)
> Maybe there's a race condition where the tests get run immediately at
> startup and before we load the test entries from a slow disk?

Is there a way to check if the entries have already been added to the internal database? When I had another look at this test we have a plain `sleep(3)` in `setUp` which can indeed be very racy.
Flags: needinfo?(francois)
(In reply to Henrik Skupin (:whimboo) from comment #13)
> (In reply to François Marier [:francois] from comment #11)
> > Maybe there's a race condition where the tests get run immediately at
> > startup and before we load the test entries from a slow disk?
> 
> Is there a way to check if the entries have already been added to the
> internal database? When I had another look at this test we have a plain
> `sleep(3)` in `setUp` which can indeed be very racy.

I'm not 100% sure, but I think that they are written to disk (~/.cache/mozilla/firefox/XXXX/safebrowsing/test-*) after they are loaded in memory. So if you check for the existence of the built-in test lists, you should be able to reliably detect that the lists are loaded.

I think my comment above was wrong, the entries aren't loaded from disk, they are added manually (hard-coded entries) and then written out to disk.
Flags: needinfo?(francois)
This might indicate an improvement. Sadly I have to say that I'm not able to work on anything for the time being. If it turns out to get a larger issue, someone else has to fix it.
I think this might be a dup of bug 1400992?
Blocks: 1328634
Flags: needinfo?(hskupin)
Not really, the patch on bug 1400992 simply changed the name of the button. This failure is around way longer.

But I would suggest to close this bug given that the failure didn't appear in the last 7 months.
Status: REOPENED → RESOLVED
Closed: 8 years ago → 7 years ago
Flags: needinfo?(hskupin)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.