821973 - HTTPS Everywhere disables Firefox updates in some situations

Status: RESOLVED WONTFIX

(Toolkit :: Blocklist Policy Requests, defect)

Description

[Germanfox]

Attachment: Firefox

There is a SSL certificate issue while trying to update or download Firefox & Firefox Beta from https://download.cdn.mozilla.net
See Attachment for more Info.

Comment 1

[Shyam Mani [:fox2mike]]

Can you provide the steps to reproduce, please?

Comment 2

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

Attachment: Certificate Chain

(In reply to Shyam Mani [:fox2mike] from comment #1)
> Can you provide the steps to reproduce, please?

To see the cert error, just put https://download.cdn.mozilla.net/ in the address bar in Firefox.

I have attached the certificate chain to this bug. The problem is that "download.cdn.mozilla.net" is not listed in the subjectAltName extension of the certificate.

Comment 3

[Shyam Mani [:fox2mike]]

Brian,

You aren't supposed to download directly from there IIRC, which is why I asked for the STR. Users will not hit download.cdn.mozilla.net to download firefox. How did you come upon this URL?

Comment 4

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Shyam Mani [:fox2mike] from comment #3)
> You aren't supposed to download directly from there IIRC, which is why I
> asked for the STR. Users will not hit download.cdn.mozilla.net to download
> firefox. How did you come upon this URL?

I just got it from comment 0. Hopefully Germanfox will explain what was going on.

Comment 5

[Germanfox]

(In reply to Shyam Mani [:fox2mike] from comment #3)
> Brian,
> 
> You aren't supposed to download directly from there IIRC, which is why I
> asked for the STR. Users will not hit download.cdn.mozilla.net to download
> firefox. How did you come upon this URL?

For Example, If you visit https://www.mozilla.org/de/firefox/beta (de in my case for the German localization) and click on the download-button, you get redirected to http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/18.0b4/win32/de/Firefox%20Setup%2018.0b4.exe (similar happens in case of Firefox stable)
Also, I could not update my Firefox Beta 18 from b3 to b4 and got update errors permanently until I permitted the wrong SSL certificate which solved the update error.
Of course I must say that here in the Morning (about 14 hours ago), I had got redirected to httpS://download.cdn.mozilla.net/... It seems to me that the SSL Connection was deactivated because of this problem.

Comment 6

[Jake Maul [:jakem]]

Are you by chance using some sort of extension that detects and forces HTTPS, like HTTPS Everywhere?

download.cdn.mozilla.net is an Akamai CDN property, and is HTTP-only for cost reasons. Our websites should *never* direct you to https://download.cdn.mozilla.net, and if they do then there's a mistake somewhere. This applies to new installer downloads and to updates.

There is a separate, SSL-aware CDN that we use for distributing new installers... https://download-installer.cdn.mozilla.net/. This has a proper cert, and we do direct traffic to it.


When you visit https://www.mozilla.org/de/firefox/beta and click the download button, you should be directed to https://download.mozilla.org, which has a proper cert. That in turn should give you a 302 redirect to either http://download.cdn.mozilla.net/ or https://download-installer.cdn.mozilla.net/ (or, in *extremely* rare cases, ftp.mozilla.org).

(In reply to Germanfox from comment #5)
> For Example, If you visit https://www.mozilla.org/de/firefox/beta (de in my
> case for the German localization) and click on the download-button, you get
> redirected to
> http://download.cdn.mozilla.net/pub/mozilla.org/firefox/releases/18.0b4/
> win32/de/Firefox%20Setup%2018.0b4.exe (similar happens in case of Firefox
> stable)

That should work fine... this URL is http://, not https://.

> Also, I could not update my Firefox Beta 18 from b3 to b4 and got update
> errors permanently until I permitted the wrong SSL certificate which solved
> the update error.

That is very interesting, and somewhat disconcerting. If this is due to HTTPS Everywhere, then I can understand the problem. Otherwise, I can't see how this should happen... download.mozilla.org doesn't give out HTTPS links for updating, unless someone has misconfigured the relevant bouncer product and set the "SSL Only" flag by accident, which is highly unlikely.

> Of course I must say that here in the Morning (about 14 hours ago), I had
> got redirected to httpS://download.cdn.mozilla.net/... It seems to me that
> the SSL Connection was deactivated because of this problem.

We didn't change anything on the IT/WebOps side, at least not that I'm aware of (and I'm kinda the lead guy on that side of product delivery these days). Perhaps you were directed to download-installer rather than just download?

Comment 7

[Germanfox]

First of all, I want to thank you and appreciate the time you took to investigate this issue.

(In reply to Jake Maul [:jakem] from comment #6)
> Are you by chance using some sort of extension that detects and forces
> HTTPS, like HTTPS Everywhere?

Yes, I do. I was using https everywhere 4.0 development3.
 
> When you visit https://www.mozilla.org/de/firefox/beta and click the
> download button, you should be directed to https://download.mozilla.org,
> which has a proper cert. That in turn should give you a 302 redirect to
> either http://download.cdn.mozilla.net/ or
> https://download-installer.cdn.mozilla.net/ (or, in *extremely* rare cases,
> ftp.mozilla.org).

Exactly. I tested this again today by disabling the HTTPS redirection rule of HTTPS Everywhere for the Mozilla domains and I got a 302 redirect to http://download.cdn.mozila.net . By enabling the rule for the Mozilla domains, I got redirected again to that certificate warning page which was obviously an issue in https everywhere.
Meanwhile I realized that about 8 hours ago there was an update for https everywhere 4.0 development3, which has now set a rule to redirect https://download.cdn.mozilla.net to https://ftp.mozilla.org, since somebody else had also opened a ticket for this issue. See: https://trac.torproject.org/projects/tor/ticket/7717
So, from my point of view the problem is resolved :)

Comment 8

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Germanfox from comment #7)
> Yes, I do. I was using https everywhere 4.0 development3.

Updates are security critical. This is the second security-critical issue caused by this addon. We need to work with the HTTPS Everywhere developers (particularly Peter Eckersley) to fix their addon. We should blocklist the versions with the bug that causes this problem.

Comment 9

[Peter Eckersley]

This should have been fixed in HTTPS Everywhere 4.0development.4, which was released on the 17th.  Our ticket was here: https://trac.torproject.org/projects/tor/ticket/7717

I also just now moved the HTTPS Everywhere 4.0dev3 XPI file on our server, so that users don't install it by accident.

If you blocklist that specific version, will users be prompted or have an opportunity to autoupgrade to 4.0development.4?

Comment 10

[Jorge Villalobos [:jorgev] (he/him)]

Blocked extensions can still be updated. If the new version isn't blocked, the extension is enabled again.

Comment 11

[Brian Smith (:briansmith, :bsmith, use NEEDINFO?)]

(In reply to Peter Eckersley from comment #9)
> This should have been fixed in HTTPS Everywhere 4.0development.4, which was
> released on the 17th.  Our ticket was here:
> https://trac.torproject.org/projects/tor/ticket/7717
> 
> I also just now moved the HTTPS Everywhere 4.0dev3 XPI file on our server,
> so that users don't install it by accident.

Thanks for the quick response Peter.

Comment 12

[Peter Eckersley]

(In reply to Jorge Villalobos [:jorgev] from comment #10)
> Blocked extensions can still be updated. If the new version isn't blocked,
> the extension is enabled again.

Does firefox probe for new versions when it sees the block?  If so, I'd be in favour of blocklisting; otherwise, I'm unsure about what's best for users.

I just checked our anonymised logs.  It seems that there was a peak of 22,000 users who had installed 4.0dev3 from our experimental/development release channel; there have since been 13,000 installs of 4.0dev4, so I'm guessing maybe 12,000 ± 2,000 currently affected users and shrinking.

Comment 13

[Jorge Villalobos [:jorgev] (he/him)]

(In reply to Peter Eckersley from comment #12)
> (In reply to Jorge Villalobos [:jorgev] from comment #10)
> > Blocked extensions can still be updated. If the new version isn't blocked,
> > the extension is enabled again.
> 
> Does firefox probe for new versions when it sees the block?

It doesn't, but it's something we're planning on fixing in the future.

Comment 14

[Jorge Villalobos [:jorgev] (he/him)]

It seems blocklisting this is no longer necessary. Users should have updated to newer versions of the add-on by now.