push gear.mozilla.org live (again)

RESOLVED FIXED

Status

RESOLVED FIXED
6 years ago
5 years ago

People

(Reporter: jslater, Assigned: cturra)

Tracking

Details

(URL)

(Reporter)

Description

6 years ago
Hi all. Following up from our aborted launch of gear.mozilla.org last month, Staples has made several security-related changes and we've now gotten the green light from our own security team to re-launch.

Would you be able to get this domain live again by Wednesday?

Our full plan, for the record:
- Mozilla security verifies fixes made on the Staples URL (done)
- push gear.mozilla.org live again (no later than Wednesday)
- security team does one final check, engagement team does some live testing before announcing to the whole org
- re-launch site to everyone early next week

Thanks!
(Assignee)

Comment 1

6 years ago
John - can you please link the sec review work to this bug?
(Reporter)

Comment 2

6 years ago
Am also copying Simon who did the security review and can provide further info as needed.
Depends on: 794395

Updated

6 years ago
Assignee: server-ops → server-ops-webops
Component: Server Operations → Server Operations: Web Operations
QA Contact: jdow → cshields
(Assignee)

Updated

6 years ago
Assignee: server-ops-webops → cturra
Status: NEW → ASSIGNED
(Assignee)

Comment 3

6 years ago
John - i have enabled a "test" gear store so we can stage this before announcing prod again. you can find it at:

  gear-test.mozilla.org


at this point i have a couple concerns: 

1) since they're now only listening on https all users are going to explicitly navigate to https://... which seems a bit restrictive. what we do across our assets normally is listen on http (80/tcp), but redirect any connections that come in that way to https (443/tcp).


2) the site is returning the ErrorPage.aspx page once again (like we saw before during our initial testing).

  $ curl -Ik https://gear-test.mozilla.org
  HTTP/1.1 302 Found
  Date: Tue, 16 Oct 2012 18:10:07 GMT
  Server: Microsoft-IIS/6.0
  X-Powered-By: ASP.NET
  X-AspNet-Version: 4.0.30319
  Location: /ErrorPage.aspx
  Set-Cookie: ASP.NET_SessionId=5qku0vq5250snxhi1ic1slm2; path=/; HttpOnly
  Cache-Control: private
  Content-Type: text/html; charset=utf-8
  Content-Length: 132


3) since the digital certificate they're using on the site is a for *.corpmerchandise.com, our users are going to be prompted with an certificate warning because of the domain mismatch. 

Certificate chain
 0 s:/C=US/ST=Kansas/L=Overland Park/O=STAPLES CONTRACT & COMMERCIAL, INC./OU=Information Techology/CN=*.corpmerchandise.com
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance CA-3
 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance CA-3
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
 2 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
   i:/C=US/O=Entrust.net/OU=www.entrust.net/CPS incorp. by ref. (limits liab.)/OU=(c) 1999 Entrust.net Limited/CN=Entrust.net Secure Server Certification Authority
(Reporter)

Comment 4

6 years ago
Thanks Chris. I ran this by Staples and got:

1. Will update further by EOD today.
2. Their IT team is not sure what this is and advised it may be something on the Mozilla side.
3. Quote: "There is nothing we can do about this one, as this is our certificate. It is not a bug from our perspective."

Can we go ahead and launch gear.mozilla.org? Would prefer to be testing on that if possible.
(Assignee)

Comment 5

6 years ago
(In reply to John Slater from comment #4)
> 
> 1. Will update further by EOD today.
> 2. Their IT team is not sure what this is and advised it may be something on
> the Mozilla side.

we are doing the redirect with a DNS CNAME record the same way we pushed this site live previously. as you can see from the header details in the 'curl' i previously included, this error is being served from Staples, not us. i would be happy to provide a full http header trace of the transaction if they need further details, but this can all be tested by simply navigating to https://gear-test.mozilla.org


> 3. Quote: "There is nothing we can do about this one, as this is our
> certificate. It is not a bug from our perspective."

i agree, it's not something they can solve. however, this is going to cause issues for our users as they will received a certificate warning when the hit the site.


> Can we go ahead and launch gear.mozilla.org? Would prefer to be testing on
> that if possible.

i don't want to re-launch gear.mozilla.org yet since it had previously been announced and we might be premature traffic. i have however pushed the following so we can continue to test this and iron out any last minute details:

  https://gear-test.mozilla.org
"The certificate is only valid for the following names:
  *.corpmerchandise.com , corpmerchandise.com , stg.staplesuniform.corpmerchandise.com , stg.staples.corpmerchandise.com , stg.jpmc.corpmerchandise.com , staplesuniform.corpmerchandise.com  "

They are not using our cert.  if and when we point gear.mozilla.org to this IP, it will raise a cert error.

I asked Chris to go ahead and make the change so we can prove to Staples that it is in fact their problem.
(Reporter)

Comment 7

6 years ago
Thanks all. Corey, what should I be telling Staples re: the cert then?
(In reply to John Slater from comment #7)
> Thanks all. Corey, what should I be telling Staples re: the cert then?

You could point them to this bug - the cert they have on our store is not the cert that we gave them to use and therefore does not match gear.mozilla.org
Also, http://gear.mozilla.org does not appear to redirect to https://gear.mozilla.org
(Reporter)

Comment 10

6 years ago
Thanks all. I'm in touch with Staples and will let you know what I hear.

> i don't want to re-launch gear.mozilla.org yet since it had previously been
> announced and we might be premature traffic. 

Re: this, I'd really like to push gear.mozilla.org live. I think the odds of people visiting a site that was launched and then pulled down 3 weeks ago are fairly low, and we actually want the actual URL to be operational so we can do a smaller soft launch among the Engagement team. Please let me know if you can get that done today, as I'm at a team work week and it would be a great time to do this.

Thanks much!
(Assignee)

Comment 11

6 years ago
John - gear.mozilla.org was made available yesterday morning.

$ curl -Ik https://gear.mozilla.org
HTTP/1.1 302 Found
Date: Thu, 18 Oct 2012 15:23:07 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 4.0.30319
Location: /ErrorPage.aspx
Set-Cookie: ASP.NET_SessionId=jwm2jtbjmfsxcmjxanx5m2zi; path=/; HttpOnly
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 132
After confirming the security exceptions on https://gear.mozilla.org/ I get redirected to https://gear.mozilla.org/ErrorPage.aspx which shows the message:
"We were unable to process your request.
Nullable object must have a value"
(Assignee)

Comment 13

6 years ago
:psiinon - correct. this was my observation in comment 3 (item 2). this error page is being returned from Staples.
(Reporter)

Comment 14

6 years ago
Thanks guys. Can you check https://gear.mozilla.org again? Seems like Staples has fixed the errors on their end, but I would like your professional opinion on that!
(Assignee)

Comment 15

6 years ago
John - the "Error" page it now gone, but we're still getting a certificate warning. this is covered in detail in comment 6.
(Reporter)

Comment 16

6 years ago
Can you explain how that certificate warning would manifest itself to the user? I'm not seeing it when I go there? Is it b/c I already marked the page as trustworthy the first time I received the warning?
(In reply to John Slater from comment #16)
> Can you explain how that certificate warning would manifest itself to the
> user? I'm not seeing it when I go there? Is it b/c I already marked the page
> as trustworthy the first time I received the warning?

probably, yes..
(In reply to John Slater from comment #16)
> Can you explain how that certificate warning would manifest itself to the
> user? I'm not seeing it when I go there? Is it b/c I already marked the page
> as trustworthy the first time I received the warning?

You can either use a new browser or firefox profile to test.
Or you can remove the exception you added by going to preferences->advanced->encryption->servers and search for gear.mozilla.org and remove the exception
(Reporter)

Comment 19

6 years ago
Yep, I just confirmed in Chrome. Will follow up with Staples again...

Thanks all.
(Assignee)

Updated

6 years ago
Depends on: 807549
(Assignee)

Comment 20

6 years ago
site has been soft launched. marking bug as r/fixed.
Status: ASSIGNED → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
Component: Server Operations: Web Operations → WebOps: Other
Product: mozilla.org → Infrastructure & Operations
You need to log in before you can comment on or make changes to this bug.