[en-US] Test failures in /testSecurity/testSSLDisabledErrorPage.js

RESOLVED FIXED

Status

Mozilla QA
Mozmill Tests
P1
normal
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: mihaelav, Assigned: Andrei Eftimie)

Tracking

({regression, reproducible})

unspecified
regression, reproducible
Dependency tree / graph

Firefox Tracking Flags

(firefox34 fixed, firefox35 fixed)

Details

(Whiteboard: [mozmill-test-failure], URL)

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

3 years ago
Module:    /testSecurity/testSSLDisabledErrorPage.js
Test:      testDisableSSL
Failures:   
* The SSL error message contains disabled information - 'Firefox cannot guarantee the safety of your data on mozqa.com because it uses SSLv3, a broken security protocol.' should contain 'ssl_error_no_cypher_overlap'
* The correct SSL error title is shown - 'Unable to Connect Securely' should equal 'Secure Connection Failed'
Branches:  Beta (34.0)
Platforms: All

Reports: http://mozmill-release.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2014-11-07&to=2014-11-14&test=%2FtestSecurity%2FtestSSLDisabledErrorPage.js&func=testDisableSSL

Started to fail today
(Assignee)

Comment 1

3 years ago
Probably caused by bug 1096695
http://hg.mozilla.org/releases/mozilla-beta/rev/d585e4e50468
Blocks: 1096695
status-firefox34: --- → affected
Keywords: regression
Priority: -- → P1
(Assignee)

Comment 2

3 years ago
This is hardcoded for en-US and 34.
Assignee: nobody → andrei.eftimie
Keywords: reproducible
QA Contact: mozmill-tests → andrei.eftimie
Summary: Test failures in /testSecurity/testSSLDisabledErrorPage.js → [en-US] Test failures in /testSecurity/testSSLDisabledErrorPage.js
(Assignee)

Comment 3

3 years ago
So the only way to have this not fail our tests, is for us to also hardcode the same strings for en-US && 34.

Check http://hg.mozilla.org/releases/mozilla-beta/rev/d585e4e50468 for what we need changing.

Henrik, would this be fine with you?
I'm not that comfortable with it, but we'll only need this for 34, so this hack will only live for 6 weeks.
Flags: needinfo?(hskupin)

Comment 4

3 years ago
(In reply to Andrei Eftimie from comment #3)
> So the only way to have this not fail our tests, is for us to also hardcode
> the same strings for en-US && 34.
> 
> Check http://hg.mozilla.org/releases/mozilla-beta/rev/d585e4e50468 for what
> we need changing.
> 
> Henrik, would this be fine with you?
> I'm not that comfortable with it, but we'll only need this for 34, so this
> hack will only live for 6 weeks.

34 *and 35* which just landed, too.
(Assignee)

Comment 5

3 years ago
Created attachment 8523002 [details] [diff] [review]
hack.patch

This is for 34. I haven't yet tested it enough. Works fine on en-US, but I'd like to test other locales first. (Should be fine, but still).

Today's build for 35 is not affected yet. With Gijs info, this will fail in the following days on 35 en-US builds as well.

We can land this on Monday morning when we can also get the fix for 35.
Flags: needinfo?(hskupin)
Attachment #8523002 - Flags: review?(hskupin)
Attachment #8523002 - Flags: review?(andreea.matei)
(In reply to Andrei Eftimie from comment #3)
> So the only way to have this not fail our tests, is for us to also hardcode
> the same strings for en-US && 34.
> 
> Check http://hg.mozilla.org/releases/mozilla-beta/rev/d585e4e50468 for what
> we need changing.

FYI I will add Axel and Francesco, so they are aware of that change.

> Henrik, would this be fine with you?
> I'm not that comfortable with it, but we'll only need this for 34, so this
> hack will only live for 6 weeks.

It would be around for more than 12 weeks. But if this is only temporary, it should be fine. But maybe Axel and Flod have an opinion here too.
We're aware of that change: it's enabled only for en-US and hard-coded in Fx 34 and Fx 35, it should be fully localizable in Fx 36 (bug 1098371). I guess a temporary fix to the test is OK.
(Assignee)

Comment 8

3 years ago
For 36 this test is still passing, we'll monitor bug 1098371, and fix it appropriately when this changes.

attachment 8523002 [details] [diff] [review] is fixes the problem for both 34 and 35. I've tested various locales, and all appears to be working nicely.

We do have another, unrelated, problem with this test, as something regressed over the weekend on the server / certificate side on mozqa.com . There's bug 1100236 for that, so even if we fix this issue, the test is currently still failing because of that.
status-firefox35: --- → affected
status-firefox36: --- → unaffected
Depends on: 1100236
Comment on attachment 8523002 [details] [diff] [review]
hack.patch

Review of attachment 8523002 [details] [diff] [review]:
-----------------------------------------------------------------

::: firefox/tests/remote/testSecurity/testSSLDisabledErrorPage.js
@@ +54,5 @@
> +  // Bug 1096695
> +  // 34 en-US has this title hardcoded
> +  if (utils.appInfo.locale === "en-US") {
> +    nssFailure2title = "Unable to Connect Securely";
> +  }

If it is hard-coded for en-US, it should also be for l10n builds. So this test should fail at the moment for localized builds of the 35 and 34 branch.
Attachment #8523002 - Flags: review?(hskupin)
Attachment #8523002 - Flags: review?(andreea.matei)
Attachment #8523002 - Flags: review-
(In reply to Henrik Skupin (:whimboo) from comment #9)
> If it is hard-coded for en-US, it should also be for l10n builds. So this
> test should fail at the moment for localized builds of the 35 and 34 branch.

It's actually disabled for l10n (they display the old page, not English text).
Comment on attachment 8523002 [details] [diff] [review]
hack.patch

Review of attachment 8523002 [details] [diff] [review]:
-----------------------------------------------------------------

I see, so please update the comment so it lists Firefox 34.0 and 35.0. With that r=me.
Attachment #8523002 - Flags: review- → review+
This patch should really have been updated and landed yesterday. We get all sorts of failures now. Cosmin please quickly fix it so we can land for 35 and 34.
Flags: needinfo?(cosmin.malutan)
Sure, whimboo, but I just ran the latest beta en-US and ro locales with and without the patch and it fails every single time with the failure from bug 972770. My opinion is that we should back-port the skip patch there.
Flags: needinfo?(cosmin.malutan)
Created attachment 8524477 [details] [diff] [review]
bug1099123.patch

I updated the comment but it still fails for me with the failure in bug 972770.
Attachment #8523002 - Attachment is obsolete: true
(Assignee)

Comment 15

3 years ago
Comment on attachment 8524477 [details] [diff] [review]
bug1099123.patch

Review of attachment 8524477 [details] [diff] [review]:
-----------------------------------------------------------------

Please also change the test to use a subdomain of mozqa.com where we can see a the Security Error page, this should work. I can't properly test this ATM.
Attachment #8524477 - Flags: review-
Comment on attachment 8524477 [details] [diff] [review]
bug1099123.patch

Review of attachment 8524477 [details] [diff] [review]:
-----------------------------------------------------------------

That fix is in bug 972770, where it belongs for that failure fix. With both fixes this passes now, so I'll land this first and then the other which also enables the test.

http://hg.mozilla.org/qa/mozmill-tests/rev/0b1e03963fe9 (aurora)
http://hg.mozilla.org/qa/mozmill-tests/rev/3f7a4d9e4248 (beta)
Attachment #8524477 - Flags: review- → review+
Status: NEW → RESOLVED
Last Resolved: 3 years ago
status-firefox34: affected → fixed
status-firefox35: affected → fixed
Resolution: --- → FIXED
(In reply to Andrei Eftimie from comment #15)
> Please also change the test to use a subdomain of mozqa.com where we can see
> a the Security Error page, this should work. I can't properly test this ATM.

That will be done by the other bug 972770.
(Reporter)

Comment 18

3 years ago
Failed again with Nightly across all platforms:
http://mozmill-daily.blargon7.com/#/remote/failure?app=Firefox&branch=All&platform=All&from=2015-01-01&to=2015-01-12&test=%2FtestSecurity%2FtestSSLDisabledErrorPage.js&func=testDisableSSL
Status: RESOLVED → REOPENED
status-firefox37: --- → affected
Resolution: FIXED → ---
Please do not reopen bugs which have been marked as fixed months ago. File always a new one. Those are cheap enough. Thanks.
Status: REOPENED → RESOLVED
Last Resolved: 3 years ago3 years ago
status-firefox36: unaffected → ---
status-firefox37: affected → ---
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.