936478 - Test failure "controller.waitForPageLoad(): Timeout waiting for page loaded" in /testSecurity/testSafeBrowsingWarningPages.js

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect, P3)

Description

[Cosmin Malutan, [:cosmin-malutan]]

This failed today on Windows and Linux with Nightly en many times in a short interval, and we could't reproduce it, so it's most likely a network issue being a remote test. We will watch to see if it fails again.
I will take a ci node down to check if it reproduces. 
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa27d082ce
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa27d0ab86
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa27d1eb03
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa27d2a9d4
http://mozmill-daily.blargon7.com/#/remote/report/d17690a112360a2b3155acaa27d3378f

Comment 1

[Cosmin Malutan, [:cosmin-malutan]]

It didn't reproduce on the affected machine.

Comment 2

[Andrei Eftimie]

Some additional information.

The page where we get redirected from google:
http://safebrowsing.clients.google.com/safebrowsing/diagnostic?client=Firefox&hl=en-US&site=http://www.mozilla.org/firefox/its-an-attack.html

Looks different from the last time we saw it.
It could be that google recently (just now?) updated it we we caught some CDN refresh problems?

Comment 3

[Andrei Eftimie]

Another failure on 27, OSX, de:
http://mozmill-daily.blargon7.com/#/remote/report/2f1ca72938985fc4b989b7efcc37fa3c

Comment 4

[Andrei Eftimie]

This has failed lots of times today (~15 times)
We should try again to look into it.

Comment 5

[mario garbi]

I have tried to reproduce it on CI machines (XP and Linux) but without much luck, I will continue the investigation and hopefully I can witness a failure by observing the machines running remote testruns today.

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+2]]

I wouldn't spend too much time on it given that we hopefully can upgrade to mozmill 2 soon.

Comment 7

[mario garbi]

I tried to reproduce this failure on the failing CI machines mm-win-8-64-3 and mm-ub-1304-4 and was not able even though the failured keep happening. This might be related to a Jenkins setting since by logging on the machine and triggering a testrun_remote script won't reproduce it even once.
Judging by Henrik's comment this might be fixed in Mozmill 2.0 and will focus on other bugs.

Comment 8

[Andrei Eftimie]

Attachment: skip.patch

We need to get this skipped, way to many failures.

Comment 9

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8335867 [details] [diff] [review]
skip.patch

Review of attachment 8335867 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry, it's not applying anymore due to bug 905033.

Comment 10

[Andrei Eftimie]

Attachment: skip_2.patch

Updated skip patch to apply cleanly again :)

Comment 11

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8335930 [details] [diff] [review]
skip_2.patch

Review of attachment 8335930 [details] [diff] [review]:
-----------------------------------------------------------------

Disabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/b3fe63f861e0 (default)

Comment 12

[Andreea Matei [:AndreeaMatei]]

And aurora:
http://hg.mozilla.org/qa/mozmill-tests/rev/971bf49d48e6 (aurora)

Comment 13

[mario garbi]

I can still reproduce this on Aurora branch, I will investigate it and come with more info.

Comment 14

[Andrei Eftimie]

We've had this fail 3 times on ESR24:
http://mozmill-daily.blargon7.com/#/remote/report/94e33fe3d7ec0be6dbbfa0a702fcd47a
http://mozmill-daily.blargon7.com/#/remote/report/94e33fe3d7ec0be6dbbfa0a702fcb27f
http://mozmill-daily.blargon7.com/#/remote/report/94e33fe3d7ec0be6dbbfa0a702fd8bf0

Comment 15

[Daniel Gherasim]

Attachment: bug_936478_unskip.patch

This is a remote test and I run it today for 500 times on latest nightly with Windows 8.1. No failure at all.

My guess here is that it fails sometimes when accessing https://support.mozilla.org/en-US/kb/how-does-phishing-and-malware-protection-work?as=u&utm_source=inproduct because of the 
ssl.google-analytics service timeout. I now that's a known issue with that service.

With the new Mozmill version we can find out exactly what's failing, so let's get it unskipped and see how it's running.

This skip patch applies cleanly on nightly and aurora.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+2]]

We wont see if Ajax requests are hanging.

Comment 17

[Andreea Matei [:AndreeaMatei]]

Failed 3 times with Beta 29.0b4 en-US, at waiting for about:blank it appears, this is the line affected:
http://hg.mozilla.org/qa/mozmill-tests/file/mozilla-beta/firefox/tests/remote/testSecurity/testSafeBrowsingNotificationBar.js#l119

This is not disabled on beta anymore, I guess do to the merge.

Comment 18

[Henrik Skupin [:whimboo][⌚️UTC+2]]

So this might be because when a new tab gets opened about:blank is the first page which gets loaded, and replaced right after with the target page. Is that something we can create a minimized testcase for to get it reproduced? Wouldn't we have to wait for the new tab being opened first?

Comment 19

[Daniel Gherasim]

(In reply to Andreea Matei [:AndreeaMatei] from comment #17)
> Failed 3 times with Beta 29.0b4 en-US, at waiting for about:blank it
> appears, this is the line affected:
> http://hg.mozilla.org/qa/mozmill-tests/file/mozilla-beta/firefox/tests/
> remote/testSecurity/testSafeBrowsingNotificationBar.js#l119

I can see that this is another test then the one from this bug's title.

Comment 20

[Andreea Matei [:AndreeaMatei]]

Oh, you're right, my bad. I'll open a new bug for this and eventually make it general for "about:blank" failures.

Comment 21

[Andrei Eftimie]

Comment on attachment 8399425 [details] [diff] [review]
bug_936478_unskip.patch

Review of attachment 8399425 [details] [diff] [review]:
-----------------------------------------------------------------

Let's unskip on default and see if we're still seeing failures.

Unskipped:
http://hg.mozilla.org/qa/mozmill-tests/rev/5b5d72d07722 (default)

Comment 22

[Daniel Gherasim]

In another 500 runs on windows 8.1 and 100 on Mac OSX 10.6 I haven't seen any failure.

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+2]]

What are we waiting for here? I see no action since April 1st.

Comment 24

[Andrei Eftimie]

Transplanted:
http://hg.mozilla.org/qa/mozmill-tests/rev/9a4d1b030cd7 (mozilla-aurora)

Daniel, please backport this patch on the rest of the branches.

Comment 25

[Daniel Gherasim]

Attachment: bug_936478_unskip_beta.patch

The 1st unskip patch applies cleanly on aurora.
Here is the patch for beta, will come with another one for release.
On ESR the test is not disabled.

Testrun reports:
- UBUNTU 12.04 - 
http://mozmill-crowd.blargon7.com/#/remote/report/7ab760e27012969ae02e3b9e417707e5
http://mozmill-crowd.blargon7.com/#/remote/report/7ab760e27012969ae02e3b9e41770a53
http://mozmill-crowd.blargon7.com/#/remote/report/7ab760e27012969ae02e3b9e417711c9
- WINDOWS 8.1 -
http://mozmill-crowd.blargon7.com/#/remote/report/7ab760e27012969ae02e3b9e4177439c
http://mozmill-crowd.blargon7.com/#/remote/report/7ab760e27012969ae02e3b9e41775149
- MAC OSX 10.6 -
http://mozmill-crowd.blargon7.com/#/remote/report/7ab760e27012969ae02e3b9e417712ec
http://mozmill-crowd.blargon7.com/#/remote/report/7ab760e27012969ae02e3b9e4177175f

Comment 26

[Daniel Gherasim]

Attachment: bug_936478_unskip_release.patch

Unskip patch for release.

Comment 27

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8403803 [details] [diff] [review]
bug_936478_unskip_beta.patch

Review of attachment 8403803 [details] [diff] [review]:
-----------------------------------------------------------------

Please make sure to have the bug number mentioned in the commit messages.
Pushed:
http://hg.mozilla.org/qa/mozmill-tests/rev/993f25aa9e28 (beta)

Comment 28

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 8403805 [details] [diff] [review]
bug_936478_unskip_release.patch

Review of attachment 8403805 [details] [diff] [review]:
-----------------------------------------------------------------

Pushed:
http://hg.mozilla.org/qa/mozmill-tests/rev/d49557ae0b30 (release)

Comment 29

[Andrei Eftimie]

This still fails from time to time.

In the last 2 months we have 34 failures on the release branch:
http://mozmill-release.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2014-10-01&to=2014-11-25&test=%2FtestSecurity%2FtestSafeBrowsingWarningPages.js&func=testWarningPages

Similar failure rate on daily runs:
http://mozmill-daily.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2014-10-01&to=2014-11-25&test=%2FtestSecurity%2FtestSafeBrowsingWarningPages.js&func=testWarningPages

Comment 30

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Please do not reopen bugs which have been fixed a long time ago. Create new ones instead. They are cheap. Sadly I cannot revert the status flags.

Comment 31

[Andrei Eftimie]

(In reply to Henrik Skupin (:whimboo) from comment #30)
> Please do not reopen bugs which have been fixed a long time ago. Create new
> ones instead. They are cheap. Sadly I cannot revert the status flags.

This bug was not fixed. Should have been closed with WFM. Those flags only reflect disabled status.
Test was unskipped in April, we have failures from April up until now.

Comment 32

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Oh, I see. But please open a new bug.  This one has already about 30 comments and will only confuse.