Perma browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | That message should appear - Got false, expected true when Gecko 67 merges to Beta on 2019-03-15

RESOLVED FIXED in Firefox 67

Status

()

defect
P1
normal
RESOLVED FIXED
a month ago
29 days ago

People

(Reporter: aiakab, Assigned: aryx)

Tracking

unspecified
Firefox 68
Points:
---

Firefox Tracking Flags

(firefox-esr60 unaffected, firefox65 unaffected, firefox66 unaffected, firefox67 fixed, firefox68 fixed)

Details

Attachments

(1 attachment)

(Reporter)

Description

a month ago

[Tracking Requested - why for this release]:

Central as beta simulation :https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel%2Crunnable&revision=41345e0f6b586cb2e26095254d68e32b83ee92c7&selectedJob=234155120

Log https://treeherder.mozilla.org/logviewer.html#?job_id=234155120&repo=try

Part of that log
[task 2019-03-15T15:17:30.581Z] 15:17:30 INFO - TEST-START | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js
[task 2019-03-15T15:17:30.921Z] 15:17:30 INFO - TEST-INFO | started process screentopng
[task 2019-03-15T15:17:31.269Z] 15:17:31 INFO - TEST-INFO | screentopng: exit 0
[task 2019-03-15T15:17:31.270Z] 15:17:31 INFO - Buffered messages logged at 15:17:30
[task 2019-03-15T15:17:31.270Z] 15:17:31 INFO - Entering test bound checkUntrustedCertError
[task 2019-03-15T15:17:31.271Z] 15:17:31 INFO - Loading https://no-subject-alt-name.example.com:443 which does not have a subject specified in the certificate
[task 2019-03-15T15:17:31.271Z] 15:17:31 INFO - Loading and waiting for the cert error
[task 2019-03-15T15:17:31.271Z] 15:17:31 INFO - Clicking the exceptionDialogButton in advanced panel
[task 2019-03-15T15:17:31.271Z] 15:17:31 INFO - TEST-PASS | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | advancedButton found - {} == true -
[task 2019-03-15T15:17:31.271Z] 15:17:31 INFO - TEST-PASS | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | advancedButton should be clickable - false == false -
[task 2019-03-15T15:17:31.272Z] 15:17:31 INFO - TEST-PASS | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | badCertAdvancedPanel found - {} == true -
[task 2019-03-15T15:17:31.274Z] 15:17:31 INFO - TEST-PASS | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | badCertTechnicalInfo found - {} == true -
[task 2019-03-15T15:17:31.274Z] 15:17:31 INFO - TEST-PASS | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | "SSL_ERROR_BAD_CERT_DOMAIN" == "SSL_ERROR_BAD_CERT_DOMAIN" -
[task 2019-03-15T15:17:31.275Z] 15:17:31 INFO - TEST-PASS | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | viewCertificate found - {} == true -
[task 2019-03-15T15:17:31.276Z] 15:17:31 INFO - Buffered messages finished
[task 2019-03-15T15:17:31.276Z] 15:17:31 INFO - TEST-UNEXPECTED-FAIL | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | That message should appear - Got false, expected true
[task 2019-03-15T15:17:31.277Z] 15:17:31 INFO - Stack trace:
[task 2019-03-15T15:17:31.278Z] 15:17:31 INFO - chrome://mochikit/content/browser-test.js:test_is:1315
[task 2019-03-15T15:17:31.278Z] 15:17:31 INFO - chrome://mochitests/content/browser/browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js:checkCorrectMessages:36
[task 2019-03-15T15:17:31.279Z] 15:17:31 INFO - chrome://mochitests/content/browser/browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js:checkUntrustedCertError:49
[task 2019-03-15T15:17:31.280Z] 15:17:31 INFO - TEST-PASS | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | That message shouldn't appear -
[task 2019-03-15T15:17:31.280Z] 15:17:31 INFO - Leaving test bound checkUntrustedCertError
[task 2019-03-15T15:17:31.281Z] 15:17:31 INFO - GECKO(1071) | MEMORY STAT | vsize 639MB | residentFast 273MB | heapAllocated 97MB
[task 2019-03-15T15:17:31.282Z] 15:17:31 INFO - TEST-OK | browser/base/content/test/about/browser_aboutCertError_noSubjectAltName.js | took 399ms
[task 2019-03-15T15:17:31.282Z] 15:17:31 INFO - checking window state

Assignee: nobody → aryx.bugmail
Blocks: 1483626
Status: NEW → ASSIGNED

Comment 2

a month ago
Pushed by ncsoregi@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/b12dda048dd5
Explicitly turn on new cert error pages in browser_aboutCertError_noSubjectAltName.js to let it pass on beta r=Gijs

Unfortunately, the patch here does not solve the problem. I imported it in the latest beta sims and the failure still occurs:

https://treeherder.mozilla.org/#/jobs?repo=try&resultStatus=testfailed%2Cbusted%2Cexception&revision=1c5b6c8feb9874db9c532be6a7252b78358a4cff&selectedJob=234339474

Flags: needinfo?(aryx.bugmail)

Verified the new version works with Firefox Developer Edition.

Flags: needinfo?(aryx.bugmail)
Priority: -- → P1

Gijs, can you review this, please? The new version is completely different from the old one but Phabricator still remembers the review from the previous version. Thank you.

Flags: needinfo?(gijskruitbosch+bugs)

Comment 7

a month ago

r=me.

Johann, can you file a follow-up to fix these tests to Not Do This?

Flags: needinfo?(gijskruitbosch+bugs) → needinfo?(jhofmann)

Comment 8

a month ago
Pushed by archaeopteryx@coole-files.de:
https://hg.mozilla.org/integration/autoland/rev/995b45bd8313
Explicitly turn on new cert error pages in browser_aboutCertError_noSubjectAltName.js to let it pass on beta r=Gijs

I'm confused, not to do what? Mention Nightly? Didn't you just fix that?

Comment 10

a month ago

(In reply to Johann Hofmann [:johannh] from comment #9)

I'm confused, not to do what? Mention Nightly? Didn't you just fix that?

Not to test for strings shown at all, really...

Comment hidden (Intermittent Failures Robot)

Comment 13

a month ago
bugherder
Status: ASSIGNED → RESOLVED
Last Resolved: a month ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 68

Thank you for fixing this up and sorry for the hassle :)

As discussed on IRC, not having Fluent (or another way to dynamically get ahold of these strings) means we would effectively get rid of a lot of helpful assertions by doing this. I think that this bug was quite a special case and that folks who are updating the english in-tree copy can be bothered to update tests (speaking as the person who has done or reviewed most of the recent copy updates here).

If these sorts of incidents repeat I'm happy to reconsider.

Flags: needinfo?(jhofmann)
Comment hidden (Intermittent Failures Robot)
You need to log in before you can comment on or make changes to this bug.