Firefox updates should use HTTP because HTTPS is unreliable

NEW
Unassigned

Status

()

3 years ago
2 months ago

People

(Reporter: rbarnes, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

3 years ago
Right now, we have no way to update clients if they are unable to access HTTPS sites at all.  As I understand it, the process is something like:

* Download updates.xml over HTTPS
* If there's an update download a signed MAR file over HTTP

If we can ensure that the content returned from the initial query is signed (as the updates are), then HTTPS will basically just be there for confidentiality, and even then, it's not hiding much.  Net of that, it would just be a nice-to-have.

So we should change the updater so that if the HTTPS query fails, it will fall back to using HTTP.
With it falling back then local exploits could make it failover. Instead of adding complexity that wouldn't provide any protection I would prefer just using http and relying on the mar signing.
Daniel, using https for the update.xml doesn't provide any significant additional security over mar signing and I'd really prefer to keep things simple here. What do you think of using http for the update.xml?
Flags: needinfo?(dveditz)

Comment 3

3 years ago
I am not a security expert, but protecting the contents of update.xml seems important. At the very least, not doing so makes it easier for an attacker to implement a "denial of update" attack. Protecting other information that is transferred via the XML file also seems important, e.g., the detailsURL and openURL values.

Comment 4

3 years ago
Having HTTPS be mandatory means that the adversary has to compromise at least two systems in order to provide malicious software updates: the HTTPS server (or its cert), *and* our MAR signing key. For us at least, these systems/keys are each protected in different ways, so it does raise the bar to require compromise of both in order to compromise users. We are also very averse to single points of failure for full userbase compromise in general.

Even beyond this concern, the reduced vulnerability surface of parsing HTTPS-authenticated update XML rather than unauthenticated update XML makes it a clear winner for us to have both HTTPS in addition to signed update manifests. In other words, we feel there is real risk to giving the adversary an easy way to confuse the update XML parser.

This is definitely something we'd want a pref for so we could disable, at minimum... It also seems like something that Firefox would want to only fall back to as a last resort, if it noticed that it could not reach the update server at all. Also in that case, it should probably inform the user that something is very wrong with their network. Such users should be used to lots of HTTPS errors anyway, I'd expect. For all other users (which hopefully outnumber the minority with blocked HTTPS), it just seems bad form to silently degrade their security without warning.

Comment 5

3 years ago
Oh, additionally, omitting HTTPS also exposes the vulnerability surface of the MAR file format and parsing prior to MAR signature violation. Without HTTPS, even if the adversary can't exploit/confuse the update XML parser, they can replace the download hash in the update xml for a malicious MAR, and then attempt to exploit the code that parses+inspects the MAR file for its signature section.
(Reporter)

Comment 6

3 years ago
Mike: Thanks for chiming in.  Do I understand correctly that in your use case, you're using a different server for update pings and MAR downloads than the one Mozilla uses?  If that's the case, then it might be simplest just to make the security of these transactions depend on the value of app.update.url and the URLs provided in update.xml.  That way Mozilla could use HTTP for broader compatibility, and you could use HTTPS for higher security.

(I'm pretty sure I agree with Robert that there's not much point to fallback, and we should just use HTTP.  If we do fallback and there's an active attacker, then he can force a downgrade, so the incremental protection from HTTPS would be quite small.)

Mark: "Denial of update" is not an issue here.  A network attacker can already see when you send an HTTPS request to the appropriate server and block that request.  Even if their blocking decision is based on the content of update.xml, since the files are public he can go pull the ones that are on the server and make a decision that way.
The change would be to the Firefox app.update.url preference and other apps could continue to use an https url for that preference.

I'd prefer not doing this at all if the client had a fallback to http from https since I suspect it would be fairly simple to accomplish when compared to the mar signing exploit required.

We already inform the user to check for a new version with a link to where they can download a new version after 10 consecutive failed update check attempts.

The mar is not parsed before the mar signature is verified.

The mar hash is not check when mar signing is enabled.

Personally, I think the current notification mentioned above after consecutive failures should be sufficient for the number of users that might not be able to use https.
Changing summary since everyone agrees that there's no point in trying HTTPS first if you're going to fall back to HTTP.

Why is HTTPS unreliable? If it is how can we be on any kind of a trajectory to deprecate HTTP?

Richard's original request included an important bit that seems have dropped out:

> If we can ensure that the content returned from the initial query is signed
> (as the updates are), then HTTPS will basically just be there for confidentiality

We don't currently have a way to do that, though a couple of ways have been proposed as a standard and we're working on one of them for another feature. If the update.xml response is signed then I agree w/Richard that the confidentiality part of TLS is not as necessary for this purpose. (For the most part, anyway: the exact request URL may reveal configuration data that is not sniffable in other ways.)

Without signing we can't trust any of the meta data to be correct and we'd have to dive into what fields are supported and what the abuse potential is. The obvious candidates for lying are the detailsURL (to the extent users check the release notes from there), the download URL, and the hash.

For Firefox releases we already use http://<cdn> for the download URL so that's completely untrusted anyway. The hash is supposed to protect us here, except

> The mar hash is not checked when mar signing is enabled.

mar signing ensures that the blob we get back is one of ours, but doesn't guarantee it's the one we wanted the user to have; checking the hash would (assuming update.xml is reliable). The updater ensures that we can't be downgraded (using meta data from the signed mar file, not from update.xml) but users who were not current could be upgraded to a slightly newer, but still known vulnerable, version. Admittedly if we have an update the user is probably already on a known-vulnerable version as bad or worse as the not-newest one, but on channels with lots of updates (nightly, dev edition) there's a lot of opportunity for users to get behind and for there to have been a really bad security regression in the meantime.

Another problem: if we can't get a valid response from the update server enough times in a row we eventually (10 days?) alert the user. If we're downloading update.xml over TLS an attacker can certainly block it, but the user will get notified. If we download over an insecure channel the attacker can MITM and reply "no update available" and we'll never alert the user. Even if we sign update.xml the attacker can simply replay an old signed response that says "no update", or the update.xml for the version the user already has. We could try to address _that_ by inventing a protocol and signing content on the fly on the server end but then you have to stop and ask "why aren't we just using TLS for this?".
Flags: needinfo?(dveditz)
Summary: Firefox updates should fall back to HTTP if HTTPS fails → Firefox updates should use HTTP because HTTPS is unreliable
The hash was never considered a security feature to protect the user in our past discussions. Since the mar file on disk could easily be replaced after the hash check I would hate to go this route for a security measure since the vast majority of security bugs with app update require local access. It did let us know whether we should skip the current download and download a full update without first restarting. I'm ok with adding something to do this if the specifics of what it is protecting against are defined. If it is from downgrades or not upgrading to the latest as suggested above then the requirement for the mar to specify the version and the channel should at the least be revisited and possibly replaced with something that better meets the requirements than the channel and version.
(In reply to Robert Strong [:rstrong] (use needinfo to contact me) from comment #9)
> The hash was never considered a security feature to protect the user in our
> past discussions.

Before we signed the MARs it certainly was!

> Since the mar file on disk could easily be replaced after
> the hash check I would hate to go this route for a security measure since
> the vast majority of security bugs with app update require local access.

The "vast majority" because we've protected the most interesting and devastating attack of messing with the bits as they're downloaded. And we can't back-slide on that because that's still the biggest risk for our users. The local access bugs we're dealing with are mostly interesting in an enterprise environment. For most users if it gets to the point of having locally running malware they've already lost.

Because of signing the bits can't be completely forged, but if we're not checking hashes then the user could be fed the version they already have, or a version lower than they need but still legitimate. Granted an attacker can simply block updates, but messing with the bits gives them a way to do so without eventually triggering our warnings about update failures. It's not the worst problem in the world, but we have the tools to prevent it pretty easily.

> I'm ok with adding something to do this if the specifics of what it is
> protecting against are defined. If it is from downgrades or not upgrading
> to the latest as suggested above then the requirement for the mar to specify
> the version and the channel should at the least be revisited and possibly
> replaced with something that better meets the requirements

There are lots of steps in the update process, and different potential attacks

1. Download update.xml
2. Download the MAR
3. ????
4. update happens

The worries in the first two steps are things like
* did the user connect to the legit update site?
* did it download legitimate update.xml information?
* did it download the legitimate MAR?
and the presumed adversary lives on the network -- either some local Starbucks Bandit, or some government as in Snowden's documents. "Success" for us against the above is for the user to be notified an update is available, and to have downloaded exactly the version we intended for them to download.

Once the right bits are on the user's machine they are safe from that network attacker. There's some time, though, between the download and applying the update (represented by "????" above). Is there a local attacker? who knows. In most cases that's an uninteresting question because we can't trust anything at that point.

One case that is interesting is if the _user_ is the attacker, trying to take advantage of the maintenance service on windows to gain privilege on a locked down machine. It would be nice if we only upgraded to the exact version we intended the user to have, but there's no reliable place we can store that information: the download was performed at the user privilege level, and such metadata could be altered by the user or a user-privileged malware. Inside the signed MAR is reliable, but when we built it we only knew what version it _is_, not what version the user needed at some future time. Checking the signature for authenticity, the channel for appropriateness, and making sure the version is at least higher than what the user currently has is the best we can do against that hypothetical local attacker. It prevents disasters.

[Another approach would be to have the maintenance service make a network connection back to our update servers and validate that the local MAR is still the right one. Then we could check for the exact right version, at the cost of not allowing updates when you're offline.]

Channel and Version is good enough against a locally malicious user. They can already turn off updates, not much we can do about that, so our main worry is to prevent them from using the maintenance server to downgrade Firefox or install some non-Firefox.

We can do better and have a higher bar against network attackers -- force them to completely deny the connection (which we can detect) rather than feed us misinformation.
You need to log in before you can comment on or make changes to this bug.