864300 - EV certificate checks are failing for "https://addons.mozilla.org/licenses/5.txt"

Status: RESOLVED FIXED

(Mozilla QA Graveyard :: Mozmill Tests, defect, P2)

Description

[Andrei Eftimie]

Failures across all platforms in the Security tests.

Looks like a change in Firefox, should probably skip this fast, then issue a fix.

Affected tests:
/testSecurity/testGreenLarry.js
http://mozmill-ci.blargon7.com/#/functional/failure?branch=All&platform=All&from=2013-04-15&to=2013-04-22&test=%2FtestSecurity%2FtestGreenLarry.js&func=testGreenLarry.js%3A%3AtestLarryGreen

/testSecurity/testSecurityNotification.js
http://mozmill-ci.blargon7.com/#/functional/failure?branch=23.0&platform=All&from=2013-04-15&to=2013-04-22&test=%2FtestSecurity%2FtestSecurityNotification.js&func=testSecurityNotification.js%3A%3AtestSecNotification

/testSecurity/testSecurityInfoViaMoreInformation.js
http://mozmill-ci.blargon7.com/#/functional/failure?branch=23.0&platform=All&from=2013-04-15&to=2013-04-22&test=%2FtestSecurity%2FtestSecurityInfoViaMoreInformation.js&func=testSecurityInfoViaMoreInformation.js%3A%3AtestSecurityInfoViaMoreInformation

/testSecurity/testSubmitUnencryptedInfoWarning.js
http://mozmill-ci.blargon7.com/#/functional/failure?branch=23.0&platform=All&from=2013-04-15&to=2013-04-22&test=%2FtestSecurity%2FtestSubmitUnencryptedInfoWarning.js&func=testSubmitUnencryptedInfoWarning.js%3A%3AtestSubmitUnencryptedInfoWarning

/testSecurity/testMixedContentPage.js
http://mozmill-ci.blargon7.com/#/functional/failure?branch=23.0&platform=All&from=2013-04-15&to=2013-04-22&test=%2FtestSecurity%2FtestMixedContentPage.js&func=testMixedContentPage.js%3A%3AtestMixedContentPage

Comment 1

[Andreea Matei [:AndreeaMatei]]

I'm not reproducing them locally by running the whole Security folder.
What I see is that all 3 tests use this page: https://addons.mozilla.org/licenses/5.txt

The reports don't show the failure over testSubmitUnencryptedInfoWarning.js and testMixedContentPage.js.

We should skip them today if we don't find the cause.

Comment 2

[Andrei Eftimie]

Update

Only these 3 are current failures:
/testSecurity/testGreenLarry.js
/testSecurity/testSecurityNotification.js
/testSecurity/testSecurityInfoViaMoreInformation.js

Comment 3

[Andrei Eftimie]

Attachment: skip patch

Skip patch attached for the affected tests

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Comment on attachment 740264 [details] [diff] [review]
skip patch

Review of attachment 740264 [details] [diff] [review]:
-----------------------------------------------------------------

This misses the changes to the manifest files.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Might be a change in Firefox or something broken with the EV certificate on AMO.

Comment 6

[Andrei Eftimie]

Attachment: skip patch v2

Updated manifest.ini

Comment 7

[Andreea Matei [:AndreeaMatei]]

Comment on attachment 740266 [details] [diff] [review]
skip patch v2

Review of attachment 740266 [details] [diff] [review]:
-----------------------------------------------------------------

Disabled:
http://hg.mozilla.org/qa/mozmill-tests/rev/2d39f465b1a5 (default)

Comment 8

[Andrei Eftimie]

Attachment: skip patch v3

I think I messed this up :(
(double quotes in the manifest).

Just finished a testrun and it failed with a problem decoding the JSON report.

Comment 9

[Andrei Eftimie]

All seems good. I had a false alarm.

Comment 10

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Lets get this investigated ASAP, so we can figure out if it is a regression or an expected change.

Comment 11

[Andrei Eftimie]

Relevant pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=a09acc1ed635

Nothing stands out at first glance :(

Comment 12

[Andreea Matei [:AndreeaMatei]]

Have you used tinderbox builds? That would include fewer changesets if we're lucky.

Comment 13

[Andrei Eftimie]

That's how we got the pushlog mentioned in comment 11

Comment 14

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The changeset in comment 11 can't be related at all given that it is a merge from mozilla-central to inbound. Please repeat your tests and check again. Most likely a hick-up with changeset ids?

Comment 15

[Andrei Eftimie]

I initially used mozilla-central build to narrow that down.
Anyway, rechecked using mozilla-inbound builds and we have a winner:
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=f9405d359a22

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This is a regression of bug 787155 then. Alice already filed bug 864633 to get this fixed. So we depend on it. I would love if we would be faster in reporting such kind of regressions as our daily users. But lets wait for a fix now.

Comment 17

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Dependent bug has been fixed. So I did a backout of the skip patch:
http://hg.mozilla.org/qa/mozmill-tests/rev/ab0306e85b52

This should work now.  Marking as fixed.