1400992 - Intermittent test_safe_browsing_warning_pages.py TestSafeBrowsingWarningPages.test_warning_pages | NoSuchElementException: Unable to locate element: getMeOutButton

Status: RESOLVED FIXED

(Testing :: Firefox UI Tests, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: hskupin [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=131805892&repo=autoland

https://queue.taskcluster.net/v1/task/N-saNRQAQJOpdYgL7D_mdg/runs/0/artifacts/public/logs/live_backing.log

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Regression from bug 1363051. Francois or prathiksha, can you please take care of this regression? Thanks.

Comment 2

[Intermittent Failures Robot]

54 failures in 135 pushes (0.4 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* autoland: 54

Platform breakdown:
* linux64: 16
* windows7-32: 11
* windows10-64: 11
* osx-10-10: 8
* linux32: 8

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1400992&startday=2017-09-18&endday=2017-09-18&tree=all

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Can we please get this permanent failure fixed ASAP? It's very annoying and its very likely that the useful test will be disabled soon.

Comment 4

[:prathiksha]

(In reply to Henrik Skupin (:whimboo) from comment #3)
> Can we please get this permanent failure fixed ASAP? It's very annoying and
> its very likely that the useful test will be disabled soon.

Of course. I just got back to my computer. I'll fix it soon. :)

Comment 5

[:prathiksha]

Attachment: Bug 1400992 - Fix test_safe_browsing_warning_pages.py.

Review commit: https://reviewboard.mozilla.org/r/181272/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/181272/

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8909785 [details]
Bug 1400992 - Fix test_safe_browsing_warning_pages.py.

https://reviewboard.mozilla.org/r/181272/#review186524

::: testing/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py:89
(Diff revision 1)
>                                                           + "phishing-malware";
>              """)
>  
> -        button = self.marionette.find_element(By.ID, "reportButton")
> +        button = self.marionette.find_element(By.ID, 'seeDetailsButton')
>          button.click()
> +        time.sleep(1)

Why do you need sleep here? Please never use it in a test. Instead Wait().until() should be used if the expected element is not present right away.

::: testing/firefox-ui/tests/functional/security/test_safe_browsing_warning_pages.py:112
(Diff revision 1)
>          self.assertEquals(topic.text, "phishing-malware")
>  
>      def check_ignore_warning_button(self, unsafe_page):
> -        button = self.marionette.find_element(By.ID, 'ignoreWarningButton')
> +        button = self.marionette.find_element(By.ID, 'seeDetailsButton')
>          button.click()
> +        time.sleep(1)

Same here.

Can you please trigger try builds so that I can see what's going on here? Thanks.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8909785 [details]
Bug 1400992 - Fix test_safe_browsing_warning_pages.py.

https://reviewboard.mozilla.org/r/181272/#review186526

I would also like to see a review from Francois regarding the behavior changes.

Comment 8

[:prathiksha]

Comment on attachment 8909785 [details]
Bug 1400992 - Fix test_safe_browsing_warning_pages.py.

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/181272/diff/1-2/

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Comment on attachment 8909785 [details]
Bug 1400992 - Fix test_safe_browsing_warning_pages.py.

https://reviewboard.mozilla.org/r/181272/#review186538

r=me with passing jobs on try.

Comment 10

[François Marier [:francois]]

Comment on attachment 8909785 [details]
Bug 1400992 - Fix test_safe_browsing_warning_pages.py.

https://reviewboard.mozilla.org/r/181272/#review186602

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+2]]

To get this bug fixed I went ahead and triggered the landing of the patch.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6e57f448953d
Fix test_safe_browsing_warning_pages.py. r=francois,whimboo

Comment 13

[Intermittent Failures Robot]

483 failures in 175 pushes (2.76 failures/push) were associated with this bug yesterday.    

** This test has failed more than 200 times in the last 30 days. It should be disabled until it can be fixed. ** 

Repository breakdown:
* autoland: 258
* mozilla-inbound: 111
* mozilla-central: 54
* try: 44
* mozilla-beta: 16

Platform breakdown:
* linux64: 143
* windows7-32: 100
* windows10-64: 95
* osx-10-10: 67
* linux32: 60
* linux64-ccov: 3
* windows7-32-nightly: 2
* windows10-64-nightly: 2
* macosx64-nightly: 2
* linux64-nightly: 2
* linux32-nightly: 2
* windows7-32-devedition: 1
* windows10-64-devedition: 1
* macosx64-devedition: 1
* linux64-devedition: 1
* linux32-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1400992&startday=2017-09-19&endday=2017-09-19&tree=all

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/6e57f448953d

Comment 15

[Intermittent Failures Robot]

90 failures in 190 pushes (0.474 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* mozilla-inbound: 71
* mozilla-beta: 16
* try: 3

Platform breakdown:
* linux64: 25
* windows10-64: 17
* windows7-32: 16
* osx-10-10: 11
* linux32: 11
* windows7-32-nightly: 1
* windows7-32-devedition: 1
* windows10-64-nightly: 1
* windows10-64-devedition: 1
* macosx64-nightly: 1
* macosx64-devedition: 1
* linux64-nightly: 1
* linux64-devedition: 1
* linux32-nightly: 1
* linux32-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1400992&startday=2017-09-20&endday=2017-09-20&tree=all

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ryan, due to the early merge of m-c to m-b it looks like this would be needed there to prevent the massive failures. Should we do an uplift or wait until the next merge?

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Oh wait. This was already done with the following push, which made the test passing on beta. Thanks everyone!

https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=4bfb0b8fb6b1da31cb6c22b243bb071b6db01984&filter-searchStr=Firefox%20UI&filter-tier=1&filter-tier=2&filter-tier=3

Comment 18

[Intermittent Failures Robot]

24 failures in 199 pushes (0.121 failures/push) were associated with this bug yesterday.    

Repository breakdown:
* try: 24

Platform breakdown:
* linux64: 9
* osx-10-10: 6
* linux32: 5
* windows7-32: 3
* windows10-64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1400992&startday=2017-09-21&endday=2017-09-21&tree=all

Comment 19

[Intermittent Failures Robot]

700 failures in 943 pushes (0.742 failures/push) were associated with this bug in the last 7 days. 

This is the #1 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **   

Repository breakdown:
* autoland: 318
* mozilla-inbound: 187
* mozilla-central: 84
* try: 78
* mozilla-beta: 33

Platform breakdown:
* linux64: 205
* windows7-32: 145
* windows10-64: 131
* osx-10-10: 92
* linux32: 92
* macosx64-nightly: 5
* linux64-ccov: 5
* windows7-32-nightly: 4
* windows10-64-nightly: 4
* linux64-nightly: 4
* linux32-nightly: 4
* windows7-32-devedition: 2
* macosx64-devedition: 2
* linux64-devedition: 2
* linux32-devedition: 2
* windows10-64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1400992&startday=2017-09-18&endday=2017-09-24&tree=all

Comment 20

[Intermittent Failures Robot]

3 failures in 885 pushes (0.003 failures/push) were associated with this bug in the last 7 days.    

Repository breakdown:
* try: 3

Platform breakdown:
* windows10-64: 1
* osx-10-10: 1
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1400992&startday=2017-09-25&endday=2017-10-01&tree=all