1919544 - Consider not HTTPS-First upgrading address bar inputs with an explicit http:// scheme

Status: RESOLVED FIXED

(Core :: DOM: Security, task, P2)

Description

[Malte Jürgens [:maltejur]]

We talked about this in the last HTTPS-First bug triage, but I think we didn't come to a consensus on this yet.

When a user enters a URL with an explicit http:// scheme in the address bar, I think we should exempt that visit of the page from HTTPS-First. I think an explicit http:// scheme in the address bar is a strong indicator that the user does not want to load the page through HTTPS.

For example, consider a user first entering a domain without a scheme in the address bar. Be it a domain with a public tld, but pointing to the local configuration page of a router. Now, if that page gets broken by HTTPS-First upgrading it, and the user notices the site is broken because it got loaded through HTTPS, I would expect the first action from the user to be entering the same domain again, this time with an explicit http:// scheme. But that will have no effect, as HTTPS-First would upgrade that anyway. And I could imagine that could get very frustrating for the user, as they would a) probably know this is a problem caused by Firefox, and b) don't know how to tell Firefox not to upgrade the page load. There are ways a user could exempt this page from HTTPS-First (the identity pane and the exception button in the HTTPS-Only section in the settings), but I imagine most users won't know those.

From what I have heard, the argument for our current behavior (upgrading these inputs with an explicit scheme) is that the input could originate from the clipboard, where the user didn't actively enter the http:// scheme. This could be addressed by adding some logic to the address bar code, detecting whether a user input has happened for a given URL besides Ctrl+V / Right click+Paste. And then, depending on that, it could then be decided whether to upgrade. I could even imagine such logic already exists because of the ongoing https trimming work (bug 1850491).

From a technical perspective, this change would require a bit more than just exempting the load originating from the address bar input with an explicit scheme from HTTPS-First. The user could try to reload the site, or do another navigation on it, which would then be upgraded again. And I don't think we want that. I think the best solution for that would be adding a temporary, hidden exception permission for HTTPS-First for the principal when we encounter an address bar load with an explicit scheme. That would be similar to what we already do when encountering the site explicitly downgrading us (bug 1884921).

I want to emphasize again that, if not changed, I think this could be a big annoyance for users as soon as HTTPS-First ships by default. And from my perspective, it already is for the users encountering it in PB. Chrome, with its HTTPS Upgrades, behaves very similarly to the proposed new behavior. So I would imagine a lot of users will see any breakage they can not fix by setting an explicit scheme as a "Firefox problem", and not a "website problem".

Comment 1

[Daniel Veditz [:dveditz]]

Good point -- this does sound like it will annoy users

When a user enters a URL with an explicit http:// scheme in the address bar [...]
an explicit http:// scheme in the address bar is a strong indicator that the user does not want to load the page through HTTPS.

"enters", or "types"? I am 100% with you if the user is typing "http://", but I'm less sure if they paste it. Maybe they mean it, but more likely they just copied it from some other location that's wrong. For example, if you type a domain-looking thing like some.words.com, Slack will turn it into an http:// link. I've seen it on other apps and sites, too. Those are almost always not what anyone explicitly wanted, and wrong 90% of the time according to Firefox page load stats.

I don't know if that's an easy distinction to make or not in the URLbar code.

And I could imagine that could get very frustrating for the user, as they [...] b) don't know how to tell Firefox not to upgrade the page load.

Could we add some indication that we've manually upgraded the url? I know other permissions do that, so that if you forgot you granted a particular permission to a site there's a clue where to click to change it. But that's awkward for this. The icons that currently show up are all added to a single button that opens the site preference pane, so we can't put it there. We could maybe decorate the lock, but the lock is being merged into the shield sometime soonish. Maybe this is a terrible suggestion :-)

The user could try to reload the site, or do another navigation on it, which would then be upgraded again. And I don't think we want that. I think the best solution for that would be adding a temporary, hidden exception permission for HTTPS-First

I agree we'd have to add a temporary exception for that case. "Hidden" exception? What if the user changes their minds and wants to clear it? I'm unfamiliar with what you did for bug 1884921 so I'll look over there.

I want to emphasize again that, if not changed, I think this could be a big annoyance for users as soon as HTTPS-First ships by default

I'll emphasize my agreement

Comment 2

[Frederik Braun [:freddy]]

I agree. We should implement this suggestion.
The necessary information should be already in code: We are recording whether a URL was typed into the address bar without a scheme wasSchemeless (should be false then) and we can see from the URL that the scheme is http. I think someone who knows where, that might even be relatively easy to implement...

Comment 3

[Simon Friedberger (:simonf)]

I like this idea but should we consider making the other option for preventing the upgrade more discoverable?
Making people go to the settings to add an exception is too much work. And it seems like the identity panel isn't even always offering the option to downgrade. If we had a simple button "Try insecure HTTP instead" in security panel that might be enough.

But maybe that doesn't work if we remove the lock icon.

I would prefer a different heuristic than "was it schemeless" because that might indeed skip too many upgrades. Manually editing the scheme would be a clear sign. What about if somebody pastes the same http link again?

Comment 4

[Malte Jürgens [:maltejur]]

Attachment: Screenshot_20240326_152318.png

(In reply to Daniel Veditz [:dveditz] from comment #1)

"enters", or "types"? I am 100% with you if the user is typing "http://", but I'm less sure if they paste it. Maybe they mean it, but more likely they just copied it from some other location that's wrong. For example, if you type a domain-looking thing like some.words.com, Slack will turn it into an http:// link. I've seen it on other apps and sites, too. Those are almost always not what anyone explicitly wanted, and wrong 90% of the time according to Firefox page load stats.

I don't know if that's an easy distinction to make or not in the URLbar code.

As mentioned in my original comment, it would be nice to have a more "advanced" way of determining if the user actually typed the "http://", or just pasted it. The current browser.urlbar.trimHttps (enabled in Nightly) does make some distinctions based on user input, mainly to determine whether to display the scheme or not while editing. But I don't think that maps well to what we want to do here. If we do want a more advanced detection of if the user has edited the scheme or not, we could ask the address bar team for help.

But generally, if such a detection logic makes this way too complex, I also don't think it would be terrible to not have any special logic, and just look at the wasSchemeless we already have. In the end, even when just pasting a http://example.com and hitting enter, that can still be interpreted as the user explicitly wanting to load example.com through HTTP.

Could we add some indication that we've manually upgraded the url? [..] We could maybe decorate the lock, but the lock is being merged into the shield sometime soonish. Maybe this is a terrible suggestion :-)

This sounds like a good idea to me. We could display some sort of small arrow pointing up in the corner of the lock icon (symbolizing an upgrade) every time, we currently would display (Upgraded to HTTPS) under the panel opening when the lock icon is clicked. That would also lead users to HTTPS-Only/First opt-out UI in the lock icon panel.

I know other permissions do that, so that if you forgot you granted a particular permission to a site there's a clue where to click to change it. But that's awkward for this. The icons that currently show up are all added to a single button that opens the site preference pane, so we can't put it there.

Maybe I am also not completely understanding what you propose. Are you saying we should add a UI indicator a) when we successfully performed an upgrade, b) when we have an exception set for the site, or c) in both cases?

In any case, this idea seems to be separate from this particular bug, so let's continue discussion in Bug 1676227.

I agree we'd have to add a temporary exception for that case. "Hidden" exception? What if the user changes their minds and wants to clear it? I'm unfamiliar with what you did for bug 1884921 so I'll look over there.

Yes, this still is kind of an open question. If we added session exceptions with the type added in Bug 1886184, there would be no way for the user to override them, other than restarting the browser. If we were to expose this, we would also need to update the permission UI. This is because we only want to add an exception for HTTPS-First, not HTTPS-Only. I had some explorations on this in D205714, see the attached screenshot for what that would look like.

But at the same time, having the exceptions "only" last a session is also not ideal. For example, if the user bookmarks a site exempted through this mechanism, and then loads that bookmark after a browser restart, it will get upgraded again. We could consider just not HTTPS-First upgrading bookmark loads (with the same temporary exception), as at the point a bookmark is set, an upgrade should either have already happened, or the site does not work with HTTPS. But that would also mean that bookmarks set before HTTPS-First was enabled won't get upgraded, and also that sites which retroactively gain HTTPS support won't be upgraded. So I am not sure if I have a good solution here.

(In reply to Simon Friedberger (:simonf) from comment #3)

I like this idea but should we consider making the other option for preventing the upgrade more discoverable?
Making people go to the settings to add an exception is too much work.

As mentioned above, adding an indicator to the lock icon could help in that regard.

And it seems like the identity panel isn't even always offering the option to downgrade.

It only is displayed when HTTPS-First or -Only interacted in some way with the load, so either if it upgraded the load, or an exception is present.

If we had a simple button "Try insecure HTTP instead" in security panel that might be enough.

The problem with displaying such a button universally is that you would generally not be able to downgrade for example HSTS with it. So I am not sure that a button, which sometimes just does nothing, is better than what we currently have (just not display any HTTPS-First UI in case it wouldn't do anything). But what we could consider is changing the current dropdown to something more intuitive, like a "downgrade" button that is only displayed when the current UI would be displayed.

Comment 5

[Malte Jürgens [:maltejur]]

We should probably also keep an eye on Android when implementing this, as we currently do not set wasSchemelessInput there.

Comment 6

[Simon Friedberger (:simonf)]

Attachment: Bug 1919544 - Downgrade HTTPS by removing the S in the address bar, r?maltejur!,mak!

Comment 7

[Simon Friedberger (:simonf)]

Attachment: Bug 1919544 - Do not HTTPS-First upgrade URLs with http:// scheme r?maltejur,mak!

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c4dfee1f9194
Do not HTTPS-First upgrade URLs with http:// scheme r=mak,necko-reviewers,urlbar-reviewers,tabbrowser-reviewers

Comment 9

[Sandor Molnar[:smolnar]]

Backed out for causing bc failures @ browser_UrlbarInput_formatValue.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/ff0f608e976df78f82853646843bb33be9a08b38

Push with failures

Failure log -> TEST-UNEXPECTED-FAIL | browser/components/urlbar/tests/browser/browser_UrlbarInput_formatValue.js

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8b1748334420
Do not HTTPS-First upgrade URLs with http:// scheme r=mak,necko-reviewers,urlbar-reviewers,tabbrowser-reviewers

Comment 11

[agoloman]

https://hg.mozilla.org/mozilla-central/rev/8b1748334420