1729618 - Forcing HTTPS-only when not selected [HTTPS-first settings are invisible and confusing]

Status: RESOLVED WORKSFORME

(Core :: DOM: Security, defect, P2)

Description

[Brett]

Attachment: BOM.jpg

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0

Steps to reproduce:

Both the previous update of Firefox and the current version (92.0) seem to be forcing HTTPS-only, even though this is not selected in Settings>Privacy&Security (that is, "Don’t enable HTTPS-Only Mode" IS selected). For example, http://www.bom.gov.au/ cannot be accessed (see screen shot attached to this report).

Actual results:

http://www.bom.gov.au/ cannot be accessed (see screen shot attached to this report). It appears that firefox is trying to force a HTTPS connection which the BOM site cannot support.

Expected results:

Previous versions of Firefox allowed HTTP connection to this site. As does Microsoft Edge currently on same computer.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::DOM: Security' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Daniel Veditz [:dveditz]]

I can't reproduce that in normal browsing. I can in a "Private Browsing" window because of "https-first" but your screenshot does not appear to be a Private window.

Comment 3

[Brett]

I've done a bit of fiddling with my privacy and security settings and it seems to be something to do with the "History" section; set to "never remember history" and I get the screen as per the screen shot (i.e. can't access BOM website), set to "remember history" and it works, I go straight to BOM website. "Never remember history" is my default setting.
Please let me know if you need info on any other settings.

Comment 4

[Daniel Veditz [:dveditz]]

Ah, yes... "Never remember history" turns on "permanent private browsing". This is definitely "HTTP-first" kicking in (see the blog post I linked above). That at least solves the mystery.

Leaving aside the known problem when upgrading http://www.bom.gov.au/ to https: in particular (tracked in bug 1725646), this shows two contradictory things:

1. The browser engine is working as intended (not a "bug" per se)
2. The fact that there's a visible pref settings that "looks like" it ought to fix this behavior frustrates users even more on top of the frustration of having their site not work

There are visible prefs about "HTTPS-only" mode, which is more aggressive about upgrading every resource loaded by a page. It defaults to off and you've gone to prefs and seen that it's off in both normal and private browsing. When the https-first behavior (upgrades only the URL in the addressbar) breaks there's no way to get to the site you want to go to. Worse, people stop looking for the hidden pref because the visible prefs make it look very clearly "OFF".

[Side note: Now that we know the cause, you can fix your immediate problem by going to about:config and changing the pref dom.security.https_first_pbm to false by double-clicking on it. That doesn't fix Firefox for other people so we'll keep this bug open for the more general work]

Https-first is not perfect, and unlike https-only there's no discoverable UI to turn it off. Here's a couple of options that aren't mutually exclusive. Maybe it will spark ideas on how to simplify this.

1. Use the https-only exceptions for https-first also, and show the per-site opt-out mechanism UI in the URL bar.
2. Always give the opt-out option whenever we upgrade the URL, not just when we think we're having problems (would help current https-only bugs too, like the one with http://www.bom.gov.au/ specifically, and the 502 error one). Assumes we've also done #1
3. Simplify our options, and then reflect that in the prefs UI

Option 3 is the slowest fix because we'll need UI change approvals and time to get new strings localized.

We currently have 4 separate prefs: two prefs each for https-only and https-first in normal mode and private mode. We don't need https-first if https-only is turned on so those could be single prefs with an integer "level" rather than a boolean. Or, it's unlikely we'd want either of those in "normal" browser and NOT private browsing so you could simplify them along those lines. In practice, the current about:preferences UI has already done that so that seems pretty natural. Would take a bunch of reworking, but instead of calling it "HTTPS-only" talk more generally about replacing HTTP urls with HTTPS and offer a similar set of radio buttons, but by default it does "HTTPS-first" and then you have a checkbox that says "upgrade all sub-resources too"; or conversely it could do https-only with a checkbox for "only upgrade urls in the addressbar" or some better way of saying that.

Or could do multi-option select boxes (like the cookie options in custom tracking protection), one each for normal mode and private mode, with the three options "upgrade all", "only upgrade navigations", "never".

Comment 5

[Daniel Veditz [:dveditz]]

Apparently we do already honor https-only exceptions in https-first (half of my option 1 above). Making the manage exception button always enabled would be a very cheap change we can roll out quickly. (turning on the option in the URL bar also would be my preference, but the preferences button is way better than nothing).

Comment 6

[Francesco]

I came here after wasting time trying to figure out why I couldn't access some sites in private browsing.

Turns out that some hosts are serving different sites depending on the protocol, e.g.:

http://www.rassegnapenitenziaria.it vs https://www.rassegnapenitenziaria.it
http://mlb.mlb.com vs https://mlb.mlb.com

In the latter case, insecure mlb redirects to www.mlb.com, secure shows an error message (though no endless loop as in bug 1725646).

I think there should be a visible option in about:preferences warning that https-first can break some sites and providing a way to disable it or add exceptions.

(IMHO serving different sites based on protocol is bad practice, but Firefox should be able to handle it nonetheless)

Comment 7

[Daniel Veditz [:dveditz]]

(In reply to Daniel Veditz [:dveditz] from comment #5)

Apparently we do already honor https-only exceptions in https-first (half of my option 1 above).

I'm not seeing that behavior now. I don't know if I tested it incorrectly last Fall or if we changed something in the meantime. Here are my STR:

1. add a permanent https-only exception for http://example.com (set to "off")
2. with https-only ON, confirm that loading http://example.com does not upgrade
3. turn off https-only
4. in about:config set dom.security.https_first to true
5. open a new tab
6. try to load http://example.com in the new tab

result: upgraded to https://example.com

If I skip step 5 and try to reload http://example.com from step 2 it does not get upgraded. Maybe we save some state in the docshell? This may be why I thought we used the exception list when I tested last September

Comment 9

[Frederik Braun [:freddy]]

We addressed bugs about remembering pages that do not support HTTPS. Also, the settings should be less confusing now.