Closed
Bug 1109613
Opened 9 years ago
Closed 8 years ago
Test failure 'expertContentHeading.getNode(...) is null' in /testSecurity/testMD5HashSignature.js
Categories
(Infrastructure & Operations :: SSL Certificates, task, P1)
Infrastructure & Operations
SSL Certificates
Tracking
(Not tracked)
RESOLVED
FIXED
People
(Reporter: cosmin-malutan, Assigned: cliang)
References
()
Details
(Whiteboard: [kanban:https://webops.kanbanize.com/ctrl_board/2/2151] [mozmill-test-failure])
Attachments
(1 file)
1.00 KB,
patch
|
AndreeaMatei
:
review+
|
Details | Diff | Splinter Review |
Test: testMD5HashSignature Failure: 'expertContentHeading.getNode(...) is null' Branches: mozilla-central Platforms: All This start failing today on nightly. I could reproduce it when I ran the test alone , but it reproduced on a full testrun, this also should be skipped.
Reporter | ||
Comment 1•9 years ago
|
||
Here is the skip patch.
Attachment #8534862 -
Flags: review?(andreea.matei)
Comment 2•9 years ago
|
||
Comment on attachment 8534862 [details] [diff] [review] skip Review of attachment 8534862 [details] [diff] [review]: ----------------------------------------------------------------- I get Secure connection Failed page when running this locally or manually opening the page. Something must have changed, we need to find what. Skipping until then.
Attachment #8534862 -
Flags: review?(andreea.matei) → review+
Comment 3•9 years ago
|
||
Secure Connection Failed An error occurred during a connection to ssl-md5.mozqa.com. Cannot communicate securely with peer: no common encryption algorithm(s). (Error code: ssl_error_no_cypher_overlap) Disabled: http://hg.mozilla.org/qa/mozmill-tests/rev/87e20bae4726 (default) Cosmin, please check what's wrong here and file a dependent bug if needed.
status-firefox37:
--- → disabled
Reporter | ||
Comment 4•9 years ago
|
||
This is not a Firefox regression, Firefox simply doesn't support SSL v3.0 https://blog.mozilla.org/security/2014/10/14/the-poodle-attack-and-the-end-of-ssl-3-0/ The preference that turned that down was "security.tls.version.fallback-limit" from commit bellow: https://hg.mozilla.org/integration/mozilla-inbound/rev/35f5ec149ad5 To summarize what test does: 1 Downloads and adds the certificate from http://mozqa.com/data/firefox/security/certificates/md5/importSSL.php 2 Adds all options as trust: "trustSSL", "trustEmail", "trustObjSign" 3 Navigates to: https://ssl-md5.mozqa.com Expected: "This Connection is Untrusted" page opens and we should add the exception Actual Result: "Problems loading page" page opens with the following content: >Secure Connection Failed > >An error occurred during a connection to ssl-md5.mozqa.com. Cannot communicate securely with peer: no common encryption algorithm(s). (Error code: ssl_error_no_cypher_overlap) > > The page you are trying to view cannot be shown because the authenticity of the received data could not be verified. > Please contact the website owners to inform them of this problem. This certificate was generated in bug 882632 by Brandon but his account is disabled. Masatoshi Kimura, do you know why changing this preference causes this and what should we to keep this test for MD5 security.
Flags: needinfo?(VYV03354)
Comment 5•9 years ago
|
||
ssl-md5.mozqa.com is TLS intolerant: https://www.ssllabs.com/ssltest/analyze.html?d=ssl-md5.mozqa.com The server software should be fixed.
Flags: needinfo?(VYV03354)
Comment 6•9 years ago
|
||
So what's next here so we can get this test re-enabled? What has to be fixed server-side exactly? Can this be done by adding TLS v1.2 to the list of supported protocols? Thing is that in this specific SSL test we only want to test the MD5 cert, but don't want to fallback to TLS 1.2. Masatoshi, can you give us some more details please? Thanks.
Comment 7•9 years ago
|
||
(In reply to Henrik Skupin (:whimboo) from comment #6) > Can this be done by adding TLS v1.2 to the list of > supported protocols? The server does not have to support TLS 1.2. All it has to do is to negotiate with the known protocol version (e.g. TLS 1.1) instead of terminating the connection abruptly even if the client offered a higher protocol version. > Thing is that in this specific SSL test we only want to > test the MD5 cert, but don't want to fallback to TLS 1.2. Sorry, I couldn't parse this sentence. What does "fallback to TLS 1.2" mean?
Comment 8•9 years ago
|
||
(In reply to Henrik Skupin (:whimboo) from comment #6) > So what's next here so we can get this test re-enabled? What has to be fixed > server-side exactly? Can this be done by adding TLS v1.2 to the list of > supported protocols? Thing is that in this specific SSL test we only want to > test the MD5 cert, but don't want to fallback to TLS 1.2. Masatoshi, can you > give us some more details please? Thanks. If all you want to test is the use of the MD5 cert, then you should definitely enable TLS 1.2, because that's going to avoid hitting a lot of other issues. However, the actual issue is that the server has a bug. Firefox is saying "Hey, I support TLS 1.2 and maybe some earlier versions." The server is supposed to reply "Let's talk using TLS 1.0" but instead it barfs. This kind of bug is usually a sign that the server is using old, outdated, and probably non-secure software. What TLS/HTTP software is the server running? Note that Gecko's xpcshell test suite already has some tests that verify that MD5 certificates are not accepted and that MD5 certificate errors can be overridden.
Flags: needinfo?(hskupin)
Comment 9•9 years ago
|
||
(In reply to Brian Smith (:briansmith, :bsmith, use NEEDINFO?) from comment #8) > However, the actual issue is that the server has a bug. Firefox is saying > "Hey, I support TLS 1.2 and maybe some earlier versions." The server is > supposed to reply "Let's talk using TLS 1.0" but instead it barfs. This kind > of bug is usually a sign that the server is using old, outdated, and > probably non-secure software. > > What TLS/HTTP software is the server running? Philippe was managing all the SSL related requests lately. So he will be able to give a better and more in-depth reply to you. > Note that Gecko's xpcshell test suite already has some tests that verify > that MD5 certificates are not accepted and that MD5 certificate errors can > be overridden. Requesting additional information from Kai given that this test request came in from him. Maybe the tests you are referring here have been added recently to xpcshell?
Flags: needinfo?(kaie)
Flags: needinfo?(hskupin)
Flags: needinfo?(gozer)
Comment 10•9 years ago
|
||
(In reply to Brian Smith (:briansmith, :bsmith, use NEEDINFO?) from comment #8) > (In reply to Henrik Skupin (:whimboo) from comment #6) > > So what's next here so we can get this test re-enabled? What has to be fixed > > server-side exactly? Can this be done by adding TLS v1.2 to the list of > > supported protocols? Thing is that in this specific SSL test we only want to > > test the MD5 cert, but don't want to fallback to TLS 1.2. Masatoshi, can you > > give us some more details please? Thanks. > > If all you want to test is the use of the MD5 cert, then you should > definitely enable TLS 1.2, because that's going to avoid hitting a lot of > other issues. According to Zeus (our SSL terminating load-balancer) all TLS versions are enabled and supported. > However, the actual issue is that the server has a bug. Firefox is saying > "Hey, I support TLS 1.2 and maybe some earlier versions." The server is > supposed to reply "Let's talk using TLS 1.0" but instead it barfs. This kind > of bug is usually a sign that the server is using old, outdated, and > probably non-secure software. Would be nice to see an easy to reproduce openssl invocation (or something similar) > What TLS/HTTP software is the server running? Zeus. > Note that Gecko's xpcshell test suite already has some tests that verify > that MD5 certificates are not accepted and that MD5 certificate errors can > be overridden. $> ./cipherscan ssl-md5.mozqa.com ...... Target: ssl-md5.mozqa.com:443 prio ciphersuite protocols pfs_keysize 1 DHE-RSA-AES128-SHA TLSv1,TLSv1.1 DH,1024bits 2 DHE-RSA-AES256-SHA TLSv1,TLSv1.1 DH,1024bits 3 AES128-SHA TLSv1,TLSv1.1 4 AES256-SHA TLSv1,TLSv1.1 5 DES-CBC3-SHA TLSv1,TLSv1.1 Is it possible this is hapenning because the server can't perform TLS1.2 with an MD5 signature? I could try and just disable TLS1.2 on that virtual server to see what happens then. Otherwise, open to suggestions.
Flags: needinfo?(gozer)
Comment 11•9 years ago
|
||
(In reply to Philippe M. Chiasson (:gozer) from comment #10) > Is it possible this is hapenning because the server can't perform TLS1.2 > with an MD5 signature? Yes, it is possible. In the ClientHello, Firefox gives a list of signature algorithms that are supported, and no MD5-based signature algorithms are in that list. It is possible that the server recognizes that the MD5-based certificate won't be accepted by the browser (because the browser told it so), and thus the server effectively rejects the certificate itself before Firefox even has a chance to. > I could try and just disable TLS1.2 on that virtual server to see what > happens then. That might work, because it might cause the server to ignore the signature_algorithms extension when TLS 1.2 is not used. However, it also might not work, if the server (wrongly?) processes the signature_algorithms extension for TLS < 1.2.
Comment 12•9 years ago
|
||
(In reply to Brian Smith (:briansmith, :bsmith, use NEEDINFO?) from comment #11) > > I could try and just disable TLS1.2 on that virtual server to see what > > happens then. > > That might work, because it might cause the server to ignore the > signature_algorithms extension when TLS 1.2 is not used. However, it also > might not work, if the server (wrongly?) processes the signature_algorithms > extension for TLS < 1.2. Lets get this tested at least. Maybe we have to disable all TLS and SSL versions?
Comment 13•9 years ago
|
||
As long as this is not fixed we also have to skip it for esr31: https://hg.mozilla.org/qa/mozmill-tests/rev/c931c2307e55 Philippe, can we get comment 11 tested? Looks like we missed to follow-up on that comment in all the last months.
status-firefox38:
--- → disabled
status-firefox39:
--- → disabled
status-firefox40:
--- → disabled
status-firefox41:
--- → disabled
status-firefox-esr31:
--- → disabled
Flags: needinfo?(kaie) → needinfo?(gozer)
Comment 14•9 years ago
|
||
This needs to bounce into the webops queue...
Assignee: nobody → server-ops-webops
Flags: needinfo?(gozer)
Comment 15•9 years ago
|
||
Can someone from webops please check comment 11 so we can make progress on this bug? Thanks.
status-firefox42:
--- → disabled
status-firefox43:
--- → disabled
QA Contact: mozmill-tests → hskupin
Comment 16•9 years ago
|
||
Nearly 3 months passed by here without a reply from webops. Philippe do you know whom from webop we could ask?
Flags: needinfo?(gozer)
Comment 17•9 years ago
|
||
Let's see if Atoll can pick this one up!
Flags: needinfo?(gozer) → needinfo?(rsoderberg)
Comment 18•9 years ago
|
||
(In reply to Henrik Skupin (:whimboo) from comment #16) > Nearly 3 months passed by here without a reply from webops. I apologize that you waited 3 months unnecessarily; please do consider reaching out through non-Bugzilla means in the future, to ensure that we're aware of your request. Three months ago, the Assignee or QA Contact of this bug was altered to a non-monitored Bugzilla placeholder account. We do not use that particular approach in Bugzilla. You can bring things to our awareness by any/all of: - file a dependent bug in a WebOps queue - cc: a Webops team member - reach out to our manager on IRC - email infra-webops@mozilla.com. (In reply to Cosmin Malutan, [:cosmin-malutan] from comment #4) > To summarize what test does: > 1 Downloads and adds the certificate from > http://mozqa.com/data/firefox/security/certificates/md5/importSSL.php > 2 Adds all options as trust: "trustSSL", "trustEmail", "trustObjSign" > 3 Navigates to: https://ssl-md5.mozqa.com > Expected: "This Connection is Untrusted" page opens and we should add the > exception > Actual Result: "Problems loading page" page opens with the following content: > >Secure Connection Failed > > > >An error occurred during a connection to ssl-md5.mozqa.com. Cannot communicate securely with peer: no common encryption algorithm(s). (Error code: ssl_error_no_cypher_overlap) > > > > The page you are trying to view cannot be shown because the authenticity of the received data could not be verified. > > Please contact the website owners to inform them of this problem. This is a very helpful summary, thank you. We will use it to try and reconfigure the VIP to behave correctly once more. > This certificate was generated in bug 882632 by Brandon but his account is > disabled. Brandon is formerly of Webops, so we'll pick up from here. No other bug has been filed about this issue, so I'm claiming this bug into the Webops queue so that we can track it and respond appropriately. We are under a mandatory change freeze due to the Firefox release this week, and will not be able to make alterations to repair this issue until November 11th (six days from now). We are permitted to override this for emergency changes that interfere with the launch week or with our ability to do business as a company. If you need such an override, please let us know; otherwise, sadly, we'll need to wait until the 11th to help you with this.
status-firefox37:
disabled → ---
status-firefox38:
disabled → ---
status-firefox39:
disabled → ---
status-firefox40:
disabled → ---
status-firefox41:
disabled → ---
status-firefox42:
disabled → ---
status-firefox43:
disabled → ---
status-firefox-esr31:
disabled → ---
Component: Mozmill Tests → WebOps: SSL and Domain Names
Flags: needinfo?(rsoderberg)
Product: Mozilla QA → Infrastructure & Operations
QA Contact: hskupin → smani
Version: unspecified → other
Comment 19•9 years ago
|
||
(In reply to Richard Soderberg [:atoll] from comment #18) > I apologize that you waited 3 months unnecessarily; please do consider > reaching out through non-Bugzilla means in the future, to ensure that we're > aware of your request. Yes, please. You can always email the team at infra-webops at mozilla dot com.
Comment 20•9 years ago
|
||
Richard no problems at all. It can surely wait 6 more days given that it was stuck that long. I didn't reach out because I thought the updated QA contact would work. Anyway, I will remember and good to see some progress will be made soon. Thanks in advance.
Whiteboard: [mozmill-test-failure] → [kanban:https://webops.kanbanize.com/ctrl_board/2/2151] [mozmill-test-failure]
Assignee | ||
Comment 21•8 years ago
|
||
I've explicitly disabled TLS 1.2 and explicitly enabled SSL v2 and SSL v3 (for fallback). Can you please test again and let me know what the results are?
Flags: needinfo?(hskupin)
Assignee | ||
Updated•8 years ago
|
Flags: needinfo?(brian)
Assignee | ||
Updated•8 years ago
|
Assignee: server-ops-webops → cliang
Comment 22•8 years ago
|
||
I don't work on this stuff any more.
Assignee | ||
Updated•8 years ago
|
Flags: needinfo?(brian)
Comment 23•8 years ago
|
||
Sorry for the slow reply. I wanted to have a look at the changes before I left for Christmas holidays but wasn't able to get around them. I will get it tested shortly.
Comment 24•8 years ago
|
||
So when I run the Mozmill test for the MD5 hash signature it now works with a release build of Firefox 43.0. So it looks like that those changes made it work. But I wonder if we should only keep secure versions enabled as fallback, which would include TLS 1.0 and 1.1 but not any SSL version.
Flags: needinfo?(hskupin) → needinfo?(cliang)
Comment 25•8 years ago
|
||
I don't believe we can offer you MD5 and also TLS at the same time. I *think* that Zeus restricts that to the SSL3 protocol only. So I'd rather not make any further changes to the "ssl-md5" VIP, which is now SSL (3.0) and MD5 as originally intended, and if you want to set up a *new* endpoint for new tests, we're happy to help - just file a new bug in WebOps: SSL with a precise definition of which ciphers/protocols/etc. you want enabled and we'll see if we can make that happen. RESO FIXE per comment 24, but please feel free to reopen if the MD5 hash signature tests break again.
Status: NEW → RESOLVED
Closed: 8 years ago
Flags: needinfo?(cliang)
Resolution: --- → FIXED
Comment 26•8 years ago
|
||
Alright. Will do. Thanks for the changes then! FYI the test will still not work starting from current Firefox Beta 44.0 most likely because of the security center changes. I don't think that anything here should be server-side related.
You need to log in
before you can comment on or make changes to this bug.
Description
•