1132680 - Convert Mozmill test 'remote/testSecurity/testMD5HashSignature.js' to Marionette

Status: RESOLVED WONTFIX

(Testing :: Firefox UI Tests, defect, P3)

Description

[Barbara Miller (:galgeek)]

This bug will cover the necessary work to convert the following test into Marionette.

http://hg.mozilla.org/qa/mozmill-tests/file/default/firefox/tests/remote/testSecurity/testMD5HashSignature.js

Comment 1

[Henrik Skupin [:whimboo][⌚️UTC+1]]

This test is blocked on handling modal dialogs which needs implementation via bug 887274.

Comment 2

[Barbara Miller (:galgeek)]

The mozmill version of this test fails for me locally with the same error and apparent cause as reported in Bug 1109613 in December 2014. The mozmill test has apparently been skipped since then. It doesn't look like a simple conversion of this test is possible while Bug 1109613 remains unresolved.

Note also that Bug 1109613 comment 8 reports that Gecko's xpcshell test suite includes similar tests. Should Firefox UI Tests cover this, too?

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+1]]

I would like to check this once we can actually test it. For now lets add the dependency for our testing site.

Comment 4

[Henrik Skupin [:whimboo][⌚️UTC+1]]

We can now finally convert this test to Marionette. The only thing we have to keep in mind is that the test will currently fail for Nightly down to beta (44.0) due to the security center changes. So a bit of element tweaking should be necessary.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Actually we are not blocked on bug 887274 for handling modal dialogs. We can easily access those without the Alert class. See my comment on bug 887274 comment 14. The dialog which opens is of type:

http://mxr.mozilla.org/mozilla-central/source/security/manager/pki/resources/content/downloadcert.xul

Steps we should do here:
1. Get a simple ui window class added for this dialog in puppeteer. Keep in mind to identify the dialog via the id and not the type (which is empty)
2. Update the test to use this newly created window class

Comment 6

[Henrik Skupin [:whimboo][⌚️UTC+1]]

Btw here the lines which can be used to load the page and to work with the dialog:

> urlbar = self.browser.navbar.locationbar.urlbar
> urlbar.send_keys('http://mozqa.com/data/firefox/security/certificates/md5/importSSL.php')
> urlbar.send_keys(self.keys.RETURN)

Comment 7

[John Dorlus [:Silne30]]

Scoped this for the Outreachy Project. NIing myself on it as a reminder.

Comment 8

[John Dorlus [:Silne30]]

This test has been deprioritized and we will not be doing it for outreachy.

Comment 9

[Henrik Skupin [:whimboo][⌚️UTC+1]]

David, on bug 773477 you wrote that we have a good coverage of MD5 based certificates. That questions now if the test we had with Mozmill can simply be dropped. If that is the case we clearly don't want to work on converting it to a Marionette test. 

The test can be found here:
http://hg.mozilla.org/qa/mozmill-tests/file/default/firefox/tests/remote/testSecurity/testMD5HashSignature.js

I want to note that IT is updating the squid proxy in SCL3 and it will no longer work with MD5 certificates either. So if we need the test we would have to make sure to get it running via Taskcluster.

Comment 10

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

We have good low-level tests for certificates signed with md5. The one thing this test adds is a check that the certificate error override UI works. If we have other integration tests that cover this, we don't need this specific test.

Comment 11

[Henrik Skupin [:whimboo][⌚️UTC+1]]

(In reply to David Keeler [:keeler] (use needinfo?) from comment #10)
> this test adds is a check that the certificate error override UI works. If
> we have other integration tests that cover this, we don't need this specific
> test.

Whom could we ask about that? I do not know anything about such other integration tests you are referencing here.

Comment 12

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

testMD5HashSignature.js appears to test that actually adding a certificate error override results in the user being able to visit the page. This doesn't appear to be present in other tests in that same directory. What I would do is take the lines from https://hg.mozilla.org/qa/mozmill-tests/file/d76a2a8414e3/firefox/tests/remote/testSecurity/testMD5HashSignature.js#l47 to the end of the function and add it to https://hg.mozilla.org/qa/mozmill-tests/file/tip/firefox/tests/remote/testSecurity/testUnknownIssuer.js and then remove testMD5HashSignature.js.

Comment 13

[Henrik Skupin [:whimboo][⌚️UTC+1]]

David so you are saying that we don't need the specific md5 subdomain including the certificate anymore? Reason why I ask is because we lost any < SSLv3 support in SCL3 due to a recent ZEUS upgrade and the proxy not allowing those protocols anymore. If we would still need that we have to wait until all of our tests are running in Taskcluster.

Comment 14

[Dana Keeler (she/her) (use needinfo) [:keeler] (on leave)]

Yes, it doesn't need to be specific to md5.

Comment 15

[Sylvestre Ledru [:Sylvestre]]

Moving to p3 because no activity for at least 24 weeks.

Comment 16

[Henrik Skupin [:whimboo][⌚️UTC+1]]

The possibility to test this specific test scenario is gone. So there is nothing left on this bug.