Closed Bug 804952 Opened 11 years ago Closed 10 years ago

Setup virtual host on mozqa.com for Mozmill MD5 hash signatures check

Categories

(Mozilla QA Graveyard :: Infrastructure, defect, P2)

Product:
Mozilla QA Graveyard
Component:
Infrastructure
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: AndreeaMatei, Assigned: bburton)

References

Details

We need to have the test pages used in bug 795398 on our servers, for the Mozmill remote test which verifies MD5 hash signatures are no longer accepted. 

MozTrap: https://moztrap.mozilla.org/manage/case/1292/
Blocks: 795398
Whiteboard: [litmus-data]
Assignee: nobody → andreea.matei
Andrea, this is not a litmus-data task. Here we need a virtual host on mozqa.com which mimics what Kaie has on his own box. See bug 795398 comment 5 for what has to be done here.

Jason, it would be great if you can help us here to get this setup.
Assignee: andreea.matei → jsmith
Summary: Add test files to litmus-data repository for MD5 hash signatures check → Setup virtual host on mozqa.com for Mozmill MD5 hash signatures check
Whiteboard: [litmus-data]
Anthony - What's the priority on this?
(In reply to Jason Smith [:jsmith] from comment #2)
> Anthony - What's the priority on this?

I'm going to say P2. I'd like to have the test automated by the end of this quarter and getting the virtual host on mozqa.com is a dependency.
Priority: -- → P2
We are moving to ESX in Phonix soon. So if we can lower the amount of hosts and ports we access with Mozmill it would be great. As long as we do not have that vhost we have to access the domain from Kai. I would certainly see it fixed soon. So Jason, if you can't do it, I could step in.
(In reply to Henrik Skupin (:whimboo) from comment #4)
> We are moving to ESX in Phonix soon. So if we can lower the amount of hosts
> and ports we access with Mozmill it would be great. As long as we do not
> have that vhost we have to access the domain from Kai. I would certainly see
> it fixed soon. So Jason, if you can't do it, I could step in.

Feel free to steal if you get time to get to this first before I do.
You can't use my data (cert) if you intend to run this on your own server, because of the different hostname.

As soon as you are actually ready to setup this on your own server (let's wait until then), you must tell me the hostname that you will use, and I can create a matching CA cert and a matching server cert for you...
So I would say we create a virtual host like ssl-md5.mozqa.com. Kai, if you can tell me the commands to issue I can setup it on my own. Thanks!
It's a lot of commands.
As talked on IRC Kai will give us the commands in a bit.
You'll need certutil and pk12util from NSS, and openssl.

create and go to a new directory

certutil -d . -N
(enter twice)

certutil -d . -S -n "bug804952-test-ca" -s "CN=Bug 804952 test CA, O=Test Org, L=Test Loc, ST=Test State, C=DE" -t C,C,C -x -m 0 -w -3 -v 99 -5

type random chars as prompted

when prompted for a choice, type 5 for SSL CA

then type 9 to finish

critical: no


(the above creates your CA cert that will be valid from "3 months ago" and "valid 99 months".

Now run

certutil -d . -S -n my-cert -s "CN=REPLACE_THIS_WITH_YOUR_SERVER_HOSTNAME_LIKE_WWW_SOMEHTHING_COM, O=Test Org MD5, L=Test Loc, ST=Test State, C=DE" -c bug804952-test-ca -t p,, -m 1 -v 60 -5 -6 -Z MD5

Type chars as prompted.

First choice: 0 for Server auth, then 9 to finish. critical no.

Second choice: 1 for SSL server, then 9 to finish, critical no.

This created a cert valid from today, valid for 60 months.


certutil -d . -L

shows that you have two certs


If you need your certs in PEM format for apache, we must export the certs.

First, export the CA cert (we only need the public part, so we can produce it directly).

certutil -d . -L -n bug804952-test-ca -a > bug804952-test-ca.pem


Now we must the server cert including the private key, which requires going through an intermediate step, that involves the pkcs#12 file format:

pk12util -d . -n my-cert -o my-cert.p12

Use empty passwords (press enter when prompted).

Finally, we convert that into an unencrypted PEM file with key and cert:

openssl pkcs12 -in my-cert.p12 -out my-cert.pem -nodes 

Type enter when prompted for the password.

Now you have a text file my-cert.pem that you must open with a text editor.

Remove the section that talks about "Bug 804952 test CA" (it should be the middle section, and that cert should be identical with the one contained in file bug804952-test-ca.pem).

Copy the private key section into a new text file named my-cert.key, and remove that private key from this file.

You are left with the "my-cert" cert in that file. Save it.

You can look at your cert with
openssl x509 -in my-cert.pem -inform PEM -noout -text

It should say signature algorithm md5WithRSAEncryption


Copy files bug804952-test-ca.pem (SSLCACertificateFile), my-cert.pem (SSLCertificateFile) and my-cert.key (SSLCertificateKeyFile) to your server and reference the files in your web server's configuration file, using the indicated configuration options.


Note that above certutil commands used serial numbers "0" and "1". Any additional certs you decide to create and are signed with the same CA *must* use different serial numbers (you must keep track on your own), in order to avoid trouble.
you can find certutil and pk12util in package nss-tools on fedora or rhel.
So we need a new IP address to get a new subdomain created for this MD5 SSL host. Al, how can this be done? Who has access to the rackspace server for admin tasks? I can't remember we handed that over to someone else when left QA.
Flags: needinfo?(abillings)
Our IT department has the rackspace account. We don't have any access outside of the mozqa.com box. Jason Smith has been working on mozqa.com since I left QA but he won't have access either. You'll need to file a bug with IT and, probably, find out who their rackspace contact/admin is.
Flags: needinfo?(abillings)
Assignee: jsmith → nobody
In case you use NSS 3.14.x for following the steps from comment 10, you must set NSS_ALLOW_WEAK_SIGNATURE_ALG="1" in your environment. It will override the NSS default not to allow MD5.

Set this variable ONLY when creating the certs.

DON'T set the variable while running your tests.
Working on this
Assignee: nobody → bburton
Status: NEW → ASSIGNED
I've made the CA and md5 cert on our ssl server

[root@ssl1.private.phx1 certutil]# certutil -d . -N
Enter a password which will be used to encrypt your keys.
The password should be at least 8 characters long,
and should contain at least one non-alphabetic character.

Enter new password:
Re-enter password:
[root@ssl1.private.phx1 certutil]# certutil -d . -S -n "ssl-md5-mozqa" -s "CN=Mozilla QA, O=Mozilla, L=Mountain View, ST=California, C=US" -t C,C,C -x -m 0 -w -3 -v 99 -5

A random seed must be generated that will be used in the
creation of your key.  One of the easiest ways to create a
random seed is to use the timing of keystrokes on a keyboard.

To begin, type keys on the keyboard until this progress meter
is full.  DO NOT USE THE AUTOREPEAT FUNCTION ON YOUR KEYBOARD!


Continue typing until the progress meter is full:

|************************************************************|

Finished.  Press enter to continue:


Generating key.  This may take a few moments...

		0 - SSL Client
		1 - SSL Server
		2 - S/MIME
		3 - Object Signing
		4 - Reserved for future use
		5 - SSL CA
		6 - S/MIME CA
		7 - Object Signing CA
		Other to finish
 > 5
		0 - SSL Client
		1 - SSL Server
		2 - S/MIME
		3 - Object Signing
		4 - Reserved for future use
		5 - SSL CA
		6 - S/MIME CA
		7 - Object Signing CA
		Other to finish
 > 9
Is this a critical extension [y/N]?
N
[root@ssl1.private.phx1 certutil]# ls
cert8.db  key3.db  secmod.db

[root@ssl1.private.phx1 certutil]# export NSS_ALLOW_WEAK_SIGNATURE_ALG="1"
[root@ssl1.private.phx1 certutil]# env | grep NSS
NSS_ALLOW_WEAK_SIGNATURE_ALG=1

[root@ssl1.private.phx1 certutil]# certutil -d . -S -n ssl-md5-mozqa-com -s "CN=ssl-md5.mozqa.com, O=Mozilla, L=Mountain View, ST=California, C=US" -c ssl-md5-mozqa -t p,, -m 1 -v 60 -5 -6 -Z MD5

A random seed must be generated that will be used in the
creation of your key.  One of the easiest ways to create a
random seed is to use the timing of keystrokes on a keyboard.

To begin, type keys on the keyboard until this progress meter
is full.  DO NOT USE THE AUTOREPEAT FUNCTION ON YOUR KEYBOARD!


Continue typing until the progress meter is full:

|************************************************************|

Finished.  Press enter to continue:


Generating key.  This may take a few moments...

		0 - Server Auth
		1 - Client Auth
		2 - Code Signing
		3 - Email Protection
		4 - Timestamp
		5 - OCSP Responder
		6 - Step-up
		Other to finish
 > 0
		0 - Server Auth
		1 - Client Auth
		2 - Code Signing
		3 - Email Protection
		4 - Timestamp
		5 - OCSP Responder
		6 - Step-up
		Other to finish
 > 9
Is this a critical extension [y/N]?
no
		0 - SSL Client
		1 - SSL Server
		2 - S/MIME
		3 - Object Signing
		4 - Reserved for future use
		5 - SSL CA
		6 - S/MIME CA
		7 - Object Signing CA
		Other to finish
 > 		0 - SSL Client
		1 - SSL Server
		2 - S/MIME
		3 - Object Signing
		4 - Reserved for future use
		5 - SSL CA
		6 - S/MIME CA
		7 - Object Signing CA
		Other to finish
 > 1
		0 - SSL Client
		1 - SSL Server
		2 - S/MIME
		3 - Object Signing
		4 - Reserved for future use
		5 - SSL CA
		6 - S/MIME CA
		7 - Object Signing CA
		Other to finish
 > 9
Is this a critical extension [y/N]?
no
[root@ssl1.private.phx1 certutil]# certutil -d . -L

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

ssl-md5-mozqa                                                Cu,Cu,Cu
ssl-md5-mozqa-com                                            pu,u,u
Did the next steps

[root@ssl1.private.phx1 certutil]# pk12util -d . -n ssl-md5-mozqa-com -o ssl-md5-mozqa-com.p12
Enter password for PKCS12 file:
Re-enter password:
pk12util: PKCS12 EXPORT SUCCESSFUL
[root@ssl1.private.phx1 certutil]# ls
cert8.db  key3.db  secmod.db  ssl-md5-mozqa-com.p12
[root@ssl1.private.phx1 certutil]# openssl pkcs12 -in ssl-md5-mozqa-com.p12 -out ssl-md5-mozqa-com.pem -nodes
Enter Import Password:
MAC verified OK
[root@ssl1.private.phx1 certutil]# vim ssl-md5-mozqa-com.pem

I edited the file and split it into a .key and .crt files

I ended up with this cert

-----BEGIN CERTIFICATE-----
MIICTTCCAbagAwIBAgIBADANBgkqhkiG9w0BAQUFADBhMQswCQYDVQQGEwJVUzET
MBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMNTW91bnRhaW4gVmlldzEQMA4G
A1UEChMHTW96aWxsYTETMBEGA1UEAxMKTW96aWxsYSBRQTAeFw0xMzAxMTEyMzEz
MDZaFw0yMTA0MTEyMzEzMDZaMGExCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxp
Zm9ybmlhMRYwFAYDVQQHEw1Nb3VudGFpbiBWaWV3MRAwDgYDVQQKEwdNb3ppbGxh
MRMwEQYDVQQDEwpNb3ppbGxhIFFBMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
gQCwlYHIEdlyLIvP47Dka+R08kDFfsR+1WTywoDFiY6L/5Uif9Rmxk9FG80XVa6O
qpHErkXyc2lxtnpr6u5zK1eSRaD9BEwnCB2d1rmanO6ilcL1m+KRCMM+8DFNgflF
UCS7N7jxb6G8IH+Wabzyi/Ao25+JRQFOeDRbhPVNE6ioAwIDAQABoxUwEzARBglg
hkgBhvhCAQEEBAMCAgQwDQYJKoZIhvcNAQEFBQADgYEARuELNgIQQUCz93gCUdp/
KkHau9MpLR5L/Z0MD+7D9Y/Nneflv2SbMPEKmt/M4ZG24ovZmTCxpfTghhIZbIUQ
MibUGZOSb52I0lVaAumEXcZ8yy96PNg7Z26Sn5a0624kBN4JhGkwhX3o1J4ieght
QW5pQI5Sc7Rb6oB0sWASL4I=
-----END CERTIFICATE-----
Ran into a snag because the private key as present in the .pem doesn't appear to be RSA and I didn't realize that

[root@ssl1.private.phx1 try2]# openssl rsa -in ssl-md5.mozqa.com.key > ssl-md5.mozqa.com.rsa.key
writing RSA key
[root@ssl1.private.phx1 try2]#

That fixed me up so Zeus would take the certificate
The certificate is enabled and working in Zeus.

bburton@ironbars [09:22:21] [~]
-> % curl -v -k https://ssl-md5-mozqa-zlb.vips.scl3.mozilla.com/
* About to connect() to ssl-md5-mozqa-zlb.vips.scl3.mozilla.com port 443 (#0)
*   Trying 63.245.215.91...
* connected
* Connected to ssl-md5-mozqa-zlb.vips.scl3.mozilla.com (63.245.215.91) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-SHA
* Server certificate:
* 	 subject: C=US; ST=California; L=Mountain View; O=Mozilla; CN=ssl-md5.mozqa.com
* 	 start date: 2013-04-12 03:52:01 GMT
* 	 expire date: 2018-04-12 03:52:01 GMT
* 	 common name: ssl-md5.mozqa.com (does not match 'ssl-md5-mozqa-zlb.vips.scl3.mozilla.com')
* 	 issuer: C=US; ST=California; L=Mountain View; O=Mozilla; CN=Mozilla QA
* 	 SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> GET / HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: ssl-md5-mozqa-zlb.vips.scl3.mozilla.com
> Accept: */*
>
< HTTP/1.1 403 Forbidden
< Date: Fri, 12 Apr 2013 04:22:24 GMT
< Server: Apache
< X-Backend-Server: mozqa1.qa.scl3.mozilla.com
< Content-Length: 220
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /error/noindex.html
on this server.</p>
</body></html>
* Connection #0 to host ssl-md5-mozqa-zlb.vips.scl3.mozilla.com left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):

We'll fix up the name when we do the migration
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Andreea asked me if the new environment is complete, and it seems it isn't.

Did you make the CA certificate available that issued the server cert?

Without providing that, the test case is incomplete.
As you can see from the original test instructions, the first step is to install a CA certificate, and you didn't say how to do that with your new environment.
As pointed out by Andreea on the other bug we still need the following:

> As requested, I'm adding here the details of what's still needed for bug
> 795398. The page https://ssl-md5-mozqa-zlb.vips.scl3.mozilla.com works as
> expected, but we still need for our test to download a CA certificate, as it
> is in this page: http://kuix.de/ca/nss-test-ca.php
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
And CA and server cert must match.

(If you no longer have the CA certificate that you had used for the creation of the server cert, you'll have to redo the server certificate creation, too.)
(In reply to Kai Engert (:kaie) from comment #22)
> And CA and server cert must match.
> 
> (If you no longer have the CA certificate that you had used for the creation
> of the server cert, you'll have to redo the server certificate creation,
> too.)

The new environment isn't ready, that's why I asked Andreea to move the question to bug 811869 so that I can address it when bug 811869 is complete and the new virtual host is ready to use. 

My initial read of this bug was simply the generation of the MD5 SSL certificate and making it available as a virtual host.

Now I see there are additional requirements but I am not clear on what they are.

Can you provide more details on what exactly is meant by making the CA certificate available? 

Do you want a copy of the code at http://kuix.de/ca/nss-test-ca.php hosted in the new virtual host I am setting up? If so please provide me with a link to the source repository and setup instructions.
:kaie and I chatted on IRC and cleared up some confusion I had.

I have the CA file that's needed and will use the PHP snippet from https://bugzilla.mozilla.org/show_bug.cgi?id=795398#c5

Setting this up will happen later, as it's not strictly required to get mozqa.com migrated and other work is, but it'll be done soon after we bring mozqa.com in-house.
Status: REOPENED → ASSIGNED
A huge thanks for :kaie for all his help with the SSL stuff on this and other mozqa bugs
This is setup, but waiting on the cutover in bug 811869 to get a mozqa.com subdomain and be testable
(In reply to Brandon Burton [:solarce] from comment #26)
> This is setup, but waiting on the cutover in bug 811869 to get a mozqa.com
> subdomain and be testable

This is live and ready to test at https://ssl-md5.mozqa.com/
Andreea and Kaie, can you both please verify that the certificate is correct? Thanks.
I'm still not able to see it in order to get the download dialog, like on kaie's page:
http://kuix.de/ca/nss-test-ca.php

Brandon, is there another link for that?
(In reply to Andreea Matei [:AndreeaMatei] from comment #29)
> I'm still not able to see it in order to get the download dialog, like on
> kaie's page:
> http://kuix.de/ca/nss-test-ca.php
> 
> Brandon, is there another link for that?

You need to go to https://ssl-md5.mozqa.com/ca/

Thanks
(In reply to Brandon Burton [:solarce] from comment #30)
> You need to go to https://ssl-md5.mozqa.com/ca/

Even there I don't see the dialog. Can it be that we don't need this anymore and everything is already done via the certificate? If that's the case it would be great.
(In reply to Henrik Skupin (:whimboo) from comment #31)
> (In reply to Brandon Burton [:solarce] from comment #30)
> > You need to go to https://ssl-md5.mozqa.com/ca/
> 
> Even there I don't see the dialog. Can it be that we don't need this anymore
> and everything is already done via the certificate? If that's the case it
> would be great.

What version of Firefox are you testing with? In Firefox 20 on OSX 10.8 I get the expected prompt: http://bits.inatree.org/images/Fullscreen_5_10_13_8_37_AM_173D4BB2.png
As seen right now, the workflow to check that is different. So you have to load the php snippet from a http location. Then I can see the dialog.

Kaie, can you please check again bug 795398 comment 0 and which are the right steps for us now? I wonder if we need this PHP snippet or if it would also work if we go through the invalid cert page and add an exception. Thanks
Flags: needinfo?(kaie)
You have a chicken-egg problem.

You must load the CA before you can connect to the server.
Therefore it's a bad idea to host the CA certificate on that same server.

Host the CA certificate elsewhere.

Don't use an exception.
Flags: needinfo?(kaie)
That's what I thought. Kaie, is that PHP script you wrote somewhat secret? Or can it be hosted in a public repository? If that's possible I would like to land it in http://hg.mozilla.org/qa/testcase-data/ so it gets synced with mozqa.com like any other testcase. If that's not possible where is this PHP script hosted on mozqa.com, Brandon?
(In reply to Henrik Skupin (:whimboo) from comment #35)
> That's what I thought. Kaie, is that PHP script you wrote somewhat secret?
> Or can it be hosted in a public repository? If that's possible I would like
> to land it in http://hg.mozilla.org/qa/testcase-data/ so it gets synced with
> mozqa.com like any other testcase. If that's not possible where is this PHP
> script hosted on mozqa.com, Brandon?

You can also use https://www.mozqa.com/ca/ , it will serve the CA cert I made for generating the ssl-md5.mozqa.com certificate

The PHP script is just

[root@mozqa1.qa.scl3 ~]# cat /data/www/mozqa.com/ca/index.php
<?php

header("Content-Type: application/x-x509-ca-cert");
include("./ssl-md5-mozqa-ca.pem");

?>

And it serves .pem CA cert I made.

I suppose we could put those in the Hg repo
If you want to add these to the repo please let me know
Status: ASSIGNED → RESOLVED
Closed: 10 years ago → 10 years ago
Resolution: --- → FIXED
Brandon, yes, we just discussed this in a meeting and we would like to have it in our repository, under /data.
(In reply to Andreea Matei [:AndreeaMatei] from comment #38)
> Brandon, yes, we just discussed this in a meeting and we would like to have
> it in our repository, under /data.

Andreea, please file a new bug in Mozilla QA / Infrastructure to get this added to our testcase-data repository.
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in before you can comment on or make changes to this bug.